This adds a "workspace-hack" crate, see
[mozilla's](https://hg.mozilla.org/mozilla-central/file/3a265fdc9f33e5946f0ca0a04af73acd7e6d1a39/build/workspace-hack/Cargo.toml#l7)
for a concise explanation of why this is useful. For us in practice this
means that if I were to run all the tests (`cargo nextest r
--workspace`) and then `cargo r`, all the deps from the previous cargo
command will be reused. Before this PR it would rebuild many deps due to
resolving different sets of features for them. For me this frequently
caused long rebuilds when things "should" already be cached.
To avoid manually maintaining our workspace-hack crate, we will use
[cargo hakari](https://docs.rs/cargo-hakari) to update the build files
when there's a necessary change. I've added a step to CI that checks
whether the workspace-hack crate is up to date, and instructs you to
re-run `script/update-workspace-hack` when it fails.
Finally, to make sure that people can still depend on crates in our
workspace without pulling in all the workspace deps, we use a `[patch]`
section following [hakari's
instructions](https://docs.rs/cargo-hakari/0.9.36/cargo_hakari/patch_directive/index.html)
One possible followup task would be making guppy use our
`rust-toolchain.toml` instead of having to duplicate that list in its
config, I opened an issue for that upstream: guppy-rs/guppy#481.
TODO:
- [x] Fix the extension test failure
- [x] Ensure the dev dependencies aren't being unified by Hakari into
the main dependencies
- [x] Ensure that the remote-server binary continues to not depend on
LibSSL
Release Notes:
- N/A
---------
Co-authored-by: Mikayla <mikayla@zed.dev>
Co-authored-by: Mikayla Maki <mikayla.c.maki@gmail.com>
This PR moves the `report_assistant_event` function from the
`language_models` crate to the `language_model` crate.
This allows us to drop some dependencies on `language_models`.
Release Notes:
- N/A
Closes https://github.com/zed-industries/zed/issues/10122
Closes https://github.com/zed-industries/zed/issues/25034
When formatting buffers or reloading them after they change on disk, we
performed a diff between the buffer's current contents and the new
content. We need this diff in order preserve the positions of cursors
and other decorations when updating the buffer's text.
In order to handle changes within lines, we would previously compute a
*character-wise* diff. This was extremely expensive for large files.
This PR gets rid of the character-wise diff, and instead performs a
normal line-wise diff. Then, for certain replace hunks, we compute a
secondary word-based diff. Also, I've switched to the
[`imara-diff`](https://github.com/pascalkuthe/imara-diff) crate, instead
of `similar`.
Release Notes:
- Fixed a hang that could occur when large files were changed on disk or
formatted.
This PR extracts the `ContextEditor` to the `assistant_context_editor`
crate.
As part of this, we have decoupled the `ContextEditor` from the
`AssistantPanel`.
There is now an `AssistantPanelDelegate` that the `ContextEditor` uses
when it needs to interface with the Assistant panel.
Release Notes:
- N/A
This PR adds a new `assistant_context_editor` crate.
This will ultimately house the `ContextEditor` so that it can be
consumed by both `assistant` and `assistant2`.
For the purposes of this PR, we just introduce the crate and move some
supporting constructs to it, such as the `ContextStore`.
Release Notes:
- N/A
This PR extracts the `PromptLibrary` out of the `assistant` crate and
moves it to the `prompt_library` crate.
The `PromptLibrary` is now decoupled from the specifics of the
`AssistantPanel` and `InlineAssistant`.
Release Notes:
- N/A
This PR extracts the streaming diff implementation to its own
`streaming_diff` crate.
It was duplicated between `assistant` and `assistant2`, but their
implementations were exactly the same (and I don't see a reason why they
would need to diverge).
Release Notes:
- N/A
This PR adds a new `prompt_library` crate and extracts the `PromptStore`
and `PromptBuilder` to it.
Eventually we'll want to house the `PromptLibrary` itself in this crate,
but right now that involves untangling a few dependencies.
Release Notes:
- N/A
This PR dedupes the `AssistantSettings` so we can use the same settings
for both Assistant1 and Assistant2.
We originally forked them so we could change the Assistant2 settings
freely, but given our rollout strategy for the new Assistant, I don't
think that makes sense.
This also fixes the issue where the JSON language server would show a
"Matches multiple schemas when only one must validate" warning in
`settings.json`.
Closes#23171.
Release Notes:
- Fixed the "Matches multiple schemas when only one must validate"
warning for the `assistant` setting.
This PR factors the tool definitions out of the `assistant` crate so
that they can be shared between `assistant` and `assistant2`.
`ToolWorkingSet` now lives in `assistant_tool`. The tool definitions
themselves live in `assistant_tools`, with the exception of the
`ContextServerTool`, which has been moved to the `context_server` crate.
As part of this refactoring I needed to extract the
`ContextServerSettings` to a separate `context_server_settings` crate so
that the `extension_host`—which is referenced by the `remote_server`—can
name the `ContextServerSettings` type without pulling in some undesired
dependencies.
Release Notes:
- N/A
This PR factors the language model selector out into its own
`language_model_selector` crate so that it can be reused in
`assistant2`.
Also renamed it from `ModelSelector` to `LanguageModelSelector` to be a
bit more specific.
Release Notes:
- N/A
This PR adds a new `language_models` crate to house the various language
model providers.
By extracting the provider definitions out of `language_model`, we're
able to remove `language_model`'s dependency on `editor`, which improves
incremental compilation when changing `editor`.
Release Notes:
- N/A
This changes the `/workflow` command so that instead of emitting edits
in separate steps, the user is presented with a single tab, with an
editable diff that they can apply to the buffer.
Todo
* Assistant panel
* [x] Show a patch title and a list of changed files in a block
decoration
* [x] Don't store resolved patches as state on Context. Resolve on
demand.
* [ ] Better presentation of patches in the panel
* [ ] Show a spinner while patch is streaming in
* Patches
* [x] Preserve leading whitespace in new text, auto-indent insertions
* [x] Ensure patch title is very short, to fit better in tab
* [x] Improve patch location resolution, prefer skipping whitespace over
skipping `}`
* [x] Ensure patch edits are auto-indented properly
* [ ] Apply `Update` edits via a diff between the old and new text, to
get fine-grained edits.
* Proposed changes editor
* [x] Show patch title in the tab
* [x] Add a toolbar with an "Apply all" button
* [x] Make `open excerpts` open the corresponding location in the base
buffer (https://github.com/zed-industries/zed/pull/18591)
* [x] Add an apply button above every hunk
(https://github.com/zed-industries/zed/pull/18592)
* [x] Expand all diff hunks by default
(https://github.com/zed-industries/zed/pull/18598)
* [x] Fix https://github.com/zed-industries/zed/issues/18589
* [x] Syntax highlighting doesn't work until the buffer is edited
(https://github.com/zed-industries/zed/pull/18648)
* [x] Disable LSP interaction in Proposed Changes editor
(https://github.com/zed-industries/zed/pull/18945)
* [x] No auto-indent? (https://github.com/zed-industries/zed/pull/18984)
* Prompt
* [ ] make sure old_text is unique
Release Notes:
- N/A
---------
Co-authored-by: Marshall Bowers <elliott.codes@gmail.com>
Co-authored-by: Antonio <antonio@zed.dev>
Co-authored-by: Richard <richard@zed.dev>
Co-authored-by: Marshall <marshall@zed.dev>
Co-authored-by: Nate Butler <iamnbutler@gmail.com>
Co-authored-by: Antonio Scandurra <me@as-cii.com>
Co-authored-by: Richard Feldman <oss@rtfeldman.com>
Some notes:
- You can put the cursor on the start or end line with triple backticks,
it doesn't actually have to be inside the block.
- Placing the cursor outside of a code block does nothing.
- Code blocks are determined by counting triple backticks pairs from
either start or end of buffer, and nothing else.
- If you manually select something, the selection takes precedence over
any code blocks.
Release Notes:
- Added the ability to copy surrounding code blocks in the assistant
panel into the clipboard, or inserting them directly into the editor,
without manually selecting. Place cursor anywhere in a code block
(marked by triple backticks) and use the `assistant::CopyCode` action
(`cmd-k c` / `ctrl-k c`) to copy to the clipboard, or the
`assistant::InsertIntoEditor` action (`cmd-<` / `ctrl-<`) to insert into
editor.
---------
Co-authored-by: Thorsten Ball <mrnugget@gmail.com>
Co-authored-by: Bennet <bennet@zed.dev>
This PR adds a tool registry to hold tools that can be called by the
Assistant.
Currently we just have a `now` tool for retrieving the current datetime.
This is all behind the `assistant-tool-use` feature flag which currently
needs to be explicitly opted-in to in order for the LLM to see the
tools.
Release Notes:
- N/A
## Todo
* [x] Parse and present new XML output
* [x] Resolve new edits to buffers and anchor ranges
* [x] Surface resolution errors
* [x] Steps fail to resolve because language hasn't loaded yet
* [x] Treat empty `<symbol>` tag as None
* [x] duplicate assists when editing steps
* [x] step footer blocks can appear *below* the following message header
block
## Release Notes:
- N/A
---------
Co-authored-by: Mikayla <mikayla@zed.dev>
Co-authored-by: Peter <peter@zed.dev>
Co-authored-by: Marshall <marshall@zed.dev>
Co-authored-by: Antonio <antonio@zed.dev>
Co-authored-by: Antonio Scandurra <me@as-cii.com>
This PR fixes two issues:
1. The prompt library window didn't set an `app_id` on Linux, which
caused it to be missing the Zed logo
2. A dangling reference to the window in the Wayland client code, which
caused the prompt library window not to close. See:
https://github.com/zed-industries/zed/pull/13201
Release Notes:
- Linux: Fixed the prompt library not closing on Wayland
---------
Co-authored-by: Junkui Zhang <364772080@qq.com>
Fixes:
- [x] an issue where directories would only match by prefix, causing
both a directory and a file to be matched if in the same directory
- [x] An issue where you could not continue a file completion when
selecting a directory, as `tab` on a file would always run the command.
This effectively disabled directory sub queries.
- [x] Inconsistent rendering of files and directories in the slash
command
Release Notes:
- N/A
---------
Co-authored-by: max <max@zed.dev>
This commit proposes the addition of "context serveres" and the
underlying protocol (model context protocol). Context servers allow
simple definition of slash commands in another language and running
local on the user machines. This aims to quickly prototype new commands,
and provide a way to add personal (or company wide) customizations to
the assistant panel, without having to maintain an extension. We can
use this to reuse our existing codebase, with authenticators, etc and
easily have it provide context into the assistant panel.
As such it occupies a different design space as extensions, which I
think are
more aimed towards long-term, well maintained pieces of code that can be
easily distributed.
It's implemented as a central crate for easy reusability across the
codebase
and to easily hook into the assistant panel at all points.
Design wise there are a few pieces:
1. client.rs: A simple JSON-RPC client talking over stdio to a spawned
server. This is
very close to how LSP work and likely there could be a combined client
down the line.
2. types.rs: Serialization and deserialization client for the underlying
model context protocol.
3. protocol.rs: Handling the session between client and server.
4. manager.rs: Manages settings and adding and deleting servers from a
central pool.
A server can be defined in the settings.json as:
```
"context_servers": [
{"id": "test", "executable": "python", "args": ["-m", "context_server"]
]
```
## Quick Example
A quick example of how a theoretical backend site can look like. With
roughly 100 lines
of code (nicely generated by Claude) and a bit of decorator magic (200
lines in total), one
can come up with a framework that makes it as easy as:
```python
@context_server.slash_command(name="rot13", description="Perform a rot13 transformation")
@context_server.argument(name="input", type=str, help="String to rot13")
async def rot13(input: str) -> str:
return ''.join(chr((ord(c) - 97 + 13) % 26 + 97) if c.isalpha() else c for c in echo.lower())
```
to define a new slash_command.
## Todo:
- Allow context servers to be defined in workspace settings.
- Allow passing env variables to context_servers
Release Notes:
- N/A
---------
Co-authored-by: Marshall Bowers <elliott.codes@gmail.com>
For future reference: WIP branch of copy/pasting a mixture of images and
text: https://github.com/zed-industries/zed/tree/copy-paste-images -
we'll come back to that one after landing this one.
Release Notes:
- You can now paste images into the Assistant Panel to include them as
context. Currently works only on Mac, and with Anthropic models. Future
support is planned for more models, operating systems, and image
clipboard operations.
---------
Co-authored-by: Antonio <antonio@zed.dev>
Co-authored-by: Mikayla <mikayla@zed.dev>
Co-authored-by: Jason <jason@zed.dev>
Co-authored-by: Kyle <kylek@zed.dev>
I need this to refine our prompts on the fly as I work.
Release Notes:
- Templates for prompts driving inline transformation in editors and the
terminal can now be overridden in the `~/.config/zed/prompts/templates`
directory. This is an advanced feature, and prevents you from getting
upstream changes. It's intended for use by Zed developers.
Release Notes:
- Improved streaming git diffs to be less jumpy during inline assistant
generation
---------
Co-authored-by: Antonio Scandurra <antonio@zed.dev>
This PR makes it so we normalize the line endings for prompts to LFs
(`\n`) when we load a prompt from the library.
In some cases, prompts could end up with CRLF (`\r\n`) line endings.
When these prompts were used with the `/prompt` slash command and
summarily run, the prompt text would be converted into a rope, causing
the line endings to be normalized to LFs.
However, this would happen _after_ the ranges for the
`SlashCommandOutputSection`s were computed based on the text that still
contained the CRLFs. This would then cause these ranges to be invalid
for the text with the normalized endings, resulting in a panic when
converting them to anchors.
Fixes https://github.com/zed-industries/zed/issues/15652.
Release Notes:
- N/A
Co-authored-by: Max <max@zed.dev>
- [x] bug: setting a key doesn't update anything
- [x] show high-level text on configuration page to explain what it is
- [x] show "everything okay!" status when credentials are set
- [x] maybe: add "verify" button to check credentials
- [x] open configuration page when opening panel for first time and
nothing is configured
- [x] BUG: need to fix empty assistant panel if provider is `zed.dev`
but not logged in
Co-Authored-By: Thorsten <thorsten@zed.dev>
Release Notes:
- N/A
---------
Co-authored-by: Thorsten <thorsten@zed.dev>
Co-authored-by: Nate Butler <iamnbutler@gmail.com>
Co-authored-by: Thorsten Ball <mrnugget@gmail.com>
This UI change is behind a `ZedPro` feature flag so that it won't be
visible until we're ready to launch that service.
Release Notes:
- N/A
---------
Co-authored-by: Marshall Bowers <elliott.codes@gmail.com>
Co-authored-by: Marshall <marshall@zed.dev>
We also eliminate the `completion` crate and moved its logic into
`LanguageModelRegistry`.
Release Notes:
- N/A
---------
Co-authored-by: Nathan <nathan@zed.dev>
This adds the optional `PRESERVED_KEYS` constant to the `Settings`
trait,
which allows users of the trait to specify which keys should be written
to
the settings file, even if their current value matches the default
value.
That's useful for tagged settings that have, for example, a `"version"`
field
that should always be present in the user settings file, so we can then
reparse
the user settings based on the version.
Co-Authored-By: Thorsten <thorsten@zed.dev>
Release Notes:
- N/A
---------
Co-authored-by: Thorsten <thorsten@zed.dev>
This PR removes the feature flag for the terminal inline assistant,
making it available to everyone.
Release Notes:
- Use Ctrl+Enter to summon the inline assistant in the terminal, which
let's you generate terminal commands based on your description
([demo](https://twitter.com/thorstenball/status/1814241447383605329))
We will soon need `semantic_index` to be able to use
`CompletionProvider`. This is currently impossible due to a cyclic crate
dependency, because `CompletionProvider` lives in the `assistant` crate,
which depends on `semantic_index`.
This PR breaks the dependency cycle by extracting two crates out of
`assistant`: `language_model` and `completion`.
Only one piece of logic changed: [this
code](922fcaf5a6 (diff-3857b3707687a4d585f1200eec4c34a7a079eae8d303b4ce5b4fce46234ace9fR61-R69)).
* As of https://github.com/zed-industries/zed/pull/13276, whenever we
ask a given completion provider for its available models, OpenAI
providers would go and ask the global assistant settings whether the
user had configured an `available_models` setting, and if so, return
that.
* This PR changes it so that instead of eagerly asking the assistant
settings for this info (the new crate must not depend on `assistant`, or
else the dependency cycle would be back), OpenAI completion providers
now store the user-configured settings as part of their struct, and
whenever the settings change, we update the provider.
In theory, this change should not change user-visible behavior...but
since it's the only change in this large PR that's more than just moving
code around, I'm mentioning it here in case there's an unexpected
regression in practice! (cc @amtoaer in case you'd like to try out this
branch and verify that the feature is still working the way you expect.)
Release Notes:
- N/A
---------
Co-authored-by: Marshall Bowers <elliott.codes@gmail.com>
Note that this shouldn't have any visible user-facing behavior yet. The
feature is incomplete but we wanna merge early to avoid a long-running
branch.
Release Notes:
- N/A
---------
Co-authored-by: Nathan <nathan@zed.dev>
This pull request introduces collaboration for the assistant panel by
turning `Context` into a CRDT. `ContextStore` is responsible for sending
and applying operations, as well as synchronizing missed changes while
the connection was lost.
Contexts are shared on a per-project basis, and only the host can share
them for now. Shared contexts can be accessed via the `History` tab in
the assistant panel.
<img width="1819" alt="image"
src="https://github.com/zed-industries/zed/assets/482957/c7ae46d2-cde3-4b03-b74a-6e9b1555c154">
Please note that this doesn't implement following yet, which is
scheduled for a subsequent pull request.
Release Notes:
- N/A
This PR begins the process of making the backing infrastructure for the
`/rustdoc` command more generic such that it can be applied to
additional documentation providers.
In this PR we:
- Rename the `rustdoc` crate to `indexed_docs` as a more general-purpose
name
- Start moving rustdoc-specific functionality into
`indexed_docs::providers::rustdoc`
- Add an `IndexedDocsRegistry` to hold multiple `IndexedDocsStore`s (one
per provider)
We haven't yet removed the rustdoc-specific bits in the `DocsIndexer`.
That will follow soon.
Release Notes:
- N/A
