This PR restructures the way that tools and attachments add information
about the current project to a conversation with the assistant. Rather
than each tool call or attachment generating a new tool or system
message containing information about the project, they can all
collectively mutate a new type called a `ProjectContext`, which stores
all of the project data that should be sent to the assistant. That data
is then formatted in a single place, and passed to the assistant in one
system message.
This prevents multiple tools/attachments from including redundant
context.
Release Notes:
- N/A
---------
Co-authored-by: Kyle <kylek@zed.dev>
This fixes a tricky intermittent issue I was seeing, where failed to
chunk certain files correctly because of the way we reuse Tree-sitter
`Parser` instances across parses.
I've also accounted for leading comments in chunk boundaries, so that
items are grouped with their leading comments whenever possible when
chunking.
Finally, we've changed the `debug project index` action so that it opens
a simple debug view in a pane, instead of printing paths to the console.
This lets you click into a path and see how it was chunked.
Release Notes:
- N/A
---------
Co-authored-by: Marshall <marshall@zed.dev>
This chunking strategy uses the existing `outline` query to chunk files.
We try to find chunk boundaries that are:
* at starts or ends of lines
* nested within as few outline items as possible
Release Notes:
- N/A
Previously, a failure to embed the search query (due to a rate limit
error) would appear the same as if there were no results.
* Avoid repeatedly embedding the search query for each worktree
* Unify tasks for searching all worktree
Release Notes:
- N/A
* [x] Fixed an issue where embeddings would be assigned incorrectly to
files if a subset of embedding batches failed
* [x] Added a command to debug which paths are present in the semantic
index
* [x] Determine why so many paths are often missing from the semantic
index
* we erroring out if an embedding batch contained multiple texts that
were the same, which can happen if a worktree contains multiple copies
of the same text (e.g. a license).
Release Notes:
- N/A
---------
Co-authored-by: Marshall <marshall@zed.dev>
Co-authored-by: Nathan <nathan@zed.dev>
Co-authored-by: Kyle <kylek@zed.dev>
Co-authored-by: Kyle Kelley <rgbkrk@gmail.com>
This PR restores the `Global` trait's status as a marker trait.
This was the original intent from #7095, when it was added, that had
been lost in #9777.
The purpose of the `Global` trait is to statically convey what types can
and can't be accessed as `Global` state, as well as provide a way of
restricting access to said globals.
For example, in the case of the `ThemeRegistry` we have a private
`GlobalThemeRegistry` that is marked as `Global`:
91b3c24ed3/crates/theme/src/registry.rs (L25-L34)
We're then able to permit reading the `ThemeRegistry` from the
`GlobalThemeRegistry` via a custom getter, while still restricting which
callers are able to mutate the global:
91b3c24ed3/crates/theme/src/registry.rs (L46-L61)
Release Notes:
- N/A
fixes https://github.com/zed-industries/zed/issues/9132
By setting the app id, window managers like `sway` can apply custom
configuration like `for_window [app_id="zed"] floating enable`.
Tested using `wlprop`/`hyprctl activewindow` for wayland, `xprop` for
x11.
Release Notes:
- Zed now sets the window app id / class, which can be used e.g. in
window managers like `sway`/`i3` to define custom rules
Release Notes:
- Allow reusing the window when opening a remote project from the recent
projects picker
- Fixed an issue, which would not let you rejoin a remote project after
disconnecting from it for the first time
---------
Co-authored-by: Conrad <conrad@zed.dev>
Co-authored-by: Remco <djsmits12@gmail.com>
This is a crate only addition of a new version of the AssistantPanel.
We'll be putting this behind a feature flag while we iron out the new
experience.
Release Notes:
- N/A
---------
Co-authored-by: Nathan Sobo <nathan@zed.dev>
Co-authored-by: Antonio Scandurra <me@as-cii.com>
Co-authored-by: Conrad Irwin <conrad@zed.dev>
Co-authored-by: Marshall Bowers <elliott.codes@gmail.com>
Co-authored-by: Antonio Scandurra <antonio@zed.dev>
Co-authored-by: Nate Butler <nate@zed.dev>
Co-authored-by: Nate Butler <iamnbutler@gmail.com>
Co-authored-by: Max Brunsfeld <maxbrunsfeld@gmail.com>
Co-authored-by: Max <max@zed.dev>
This introduces semantic indexing in Zed based on chunking text from
files in the developer's workspace and creating vector embeddings using
an embedding model. As part of this, we've created an embeddings
provider trait that allows us to work with OpenAI, a local Ollama model,
or a Zed hosted embedding.
The semantic index is built by breaking down text for known
(programming) languages into manageable chunks that are smaller than the
max token size. Each chunk is then fed to a language model to create a
high dimensional vector which is then normalized to a unit vector to
allow fast comparison with other vectors with a simple dot product.
Alongside the vector, we store the path of the file and the range within
the document where the vector was sourced from.
Zed will soon grok contextual similarity across different text snippets,
allowing for natural language search beyond keyword matching. This is
being put together both for human-based search as well as providing
results to Large Language Models to allow them to refine how they help
developers.
Remaining todo:
* [x] Change `provider` to `model` within the zed hosted embeddings
database (as its currently a combo of the provider and the model in one
name)
Release Notes:
- N/A
---------
Co-authored-by: Nathan Sobo <nathan@zed.dev>
Co-authored-by: Antonio Scandurra <me@as-cii.com>
Co-authored-by: Conrad Irwin <conrad@zed.dev>
Co-authored-by: Marshall Bowers <elliott.codes@gmail.com>
Co-authored-by: Antonio <antonio@zed.dev>
Co-authored-by: Antonio <antonio@zed.dev>
Resurrected this from some assistant work I did in Spring of 2023.
- [x] Resurrect streaming responses
- [x] Use streaming responses to enable AI via Zed's servers by default
(but preserve API key option for now)
- [x] Simplify protobuf
- [x] Proxy to OpenAI on zed.dev
- [x] Proxy to Gemini on zed.dev
- [x] Improve UX for switching between openAI and google models
- We current disallow cycling when setting a custom model, but we need a
better solution to keep OpenAI models available while testing the google
ones
- [x] Show remaining tokens correctly for Google models
- [x] Remove semantic index
- [x] Delete `ai` crate
- [x] Cloud front so we can ban abuse
- [x] Rate-limiting
- [x] Fix panic when using inline assistant
- [x] Double check the upgraded `AssistantSettings` are
backwards-compatible
- [x] Add hosted LLM interaction behind a `language-models` feature
flag.
Release Notes:
- We are temporarily removing the semantic index in order to redesign it
from scratch.
---------
Co-authored-by: Antonio <antonio@zed.dev>
Co-authored-by: Antonio Scandurra <me@as-cii.com>
Co-authored-by: Thorsten <thorsten@zed.dev>
Co-authored-by: Max <max@zed.dev>
Closes#5178
Release Notes:
- Added a `file_types` setting that can be used to associate languages
with file names and file extensions. For example, to interpret all `.c`
files as C++, and files called `MyLockFile` as TOML, add the following
to `settings.json`:
```json
{
"file_types": {
"C++": ["c"],
"TOML": ["MyLockFile"]
}
}
```
As with most zed settings, this can be configured on a per-directory
basis by including a local `.zed/settings.json` file in that directory.
---------
Co-authored-by: Marshall <marshall@zed.dev>
This is just a refactor. I noticed that we now have a `project_core`
crate, which mainly contains the `Worktree` type and its private
helpers, plus the project's settings.
In this PR, I've renamed that crate to `worktree` and did some minor
simplification to its module structure. I also extracted a new
`WorktreeSettings` settings type from the `ProjectSettings`, so that the
worktree settings could live in the worktree crate. This way, the crate
is now exclusively about worktree logic.
Release Notes:
- N/A
Partially fixes#4321, since Azure OpenAI API can be converted to OpenAI
API.
Release Notes:
- Added `assistant.openai_api_url` setting to allow OpenAI API URL to be
configured.
---------
Co-authored-by: Marshall Bowers <elliott.codes@gmail.com>
This PR adds the initial support for loading extensions in Zed.
### Extensions Directory
Extensions are loaded from the extensions directory.
The extensions directory has the following structure:
```
extensions/
installed/
extension-a/
grammars/
languages/
extension-b/
themes/
manifest.json
```
The `manifest.json` file is used internally by Zed to keep track of
which extensions are installed. This file should be maintained
automatically, and shouldn't require any direct interaction with it.
Extensions can provide Tree-sitter grammars, languages, and themes.
Release Notes:
- N/A
---------
Co-authored-by: Marshall <marshall@zed.dev>
Reverts zed-industries/zed#7481
This would regress performance because we'd be using the standard
library's hash maps everywhere, so reverting for now.
This should prevent a class of bugs where one queries the wrong type of
global, which results in oddities at runtime.
Release Notes:
- N/A
---------
Co-authored-by: Marshall <marshall@zed.dev>
Co-authored-by: Marshall Bowers <elliott.codes@gmail.com>
This PR updates the casing of "OpenAI" when used in Rust identifiers to
match the [Rust naming
guidelines](https://rust-lang.github.io/api-guidelines/naming.html):
> In `UpperCamelCase`, acronyms and contractions of compound words count
as one word: use `Uuid` rather than `UUID`, `Usize` rather than `USize`
or `Stdin` rather than `StdIn`.
Release Notes:
- N/A
This PR upgrades `tree-sitter-php` to v0.21.1.
Here is the diff between our current version and this version:
d43130fd15...29a49d3a53
The primary impetus for this change is to get this change that adds the
`license` field to the `Cargo.toml`:
https://github.com/tree-sitter/tree-sitter-php/pull/193
This will silence the warning that is shown when running
`script/generate-licenses`.
Release Notes:
- Upgraded `tree-sitter-php` to v0.21.1.
