This introduces semantic indexing in Zed based on chunking text from
files in the developer's workspace and creating vector embeddings using
an embedding model. As part of this, we've created an embeddings
provider trait that allows us to work with OpenAI, a local Ollama model,
or a Zed hosted embedding.
The semantic index is built by breaking down text for known
(programming) languages into manageable chunks that are smaller than the
max token size. Each chunk is then fed to a language model to create a
high dimensional vector which is then normalized to a unit vector to
allow fast comparison with other vectors with a simple dot product.
Alongside the vector, we store the path of the file and the range within
the document where the vector was sourced from.
Zed will soon grok contextual similarity across different text snippets,
allowing for natural language search beyond keyword matching. This is
being put together both for human-based search as well as providing
results to Large Language Models to allow them to refine how they help
developers.
Remaining todo:
* [x] Change `provider` to `model` within the zed hosted embeddings
database (as its currently a combo of the provider and the model in one
name)
Release Notes:
- N/A
---------
Co-authored-by: Nathan Sobo <nathan@zed.dev>
Co-authored-by: Antonio Scandurra <me@as-cii.com>
Co-authored-by: Conrad Irwin <conrad@zed.dev>
Co-authored-by: Marshall Bowers <elliott.codes@gmail.com>
Co-authored-by: Antonio <antonio@zed.dev>
Only `CtrlC` and `CtrlBreak` signals are supported. `CtrlLogoff` and
`CtrlShutdown` is service only signal and I have not tried these yet.
`CtrlClose` occurs when terminal window is closed, but I found tokio's
`ctrl_close` does not work well, so I put comment in code.
Release Notes:
- N/A
* Allow creating channels when seeding
* Allow configuring a custom `SEED_PATH`
* Seed the database when creating/migrating it so you don't need a
separate step for this.
Release Notes:
- N/A
Co-authored-by: Antonio <antonio@zed.dev>
Resurrected this from some assistant work I did in Spring of 2023.
- [x] Resurrect streaming responses
- [x] Use streaming responses to enable AI via Zed's servers by default
(but preserve API key option for now)
- [x] Simplify protobuf
- [x] Proxy to OpenAI on zed.dev
- [x] Proxy to Gemini on zed.dev
- [x] Improve UX for switching between openAI and google models
- We current disallow cycling when setting a custom model, but we need a
better solution to keep OpenAI models available while testing the google
ones
- [x] Show remaining tokens correctly for Google models
- [x] Remove semantic index
- [x] Delete `ai` crate
- [x] Cloud front so we can ban abuse
- [x] Rate-limiting
- [x] Fix panic when using inline assistant
- [x] Double check the upgraded `AssistantSettings` are
backwards-compatible
- [x] Add hosted LLM interaction behind a `language-models` feature
flag.
Release Notes:
- We are temporarily removing the semantic index in order to redesign it
from scratch.
---------
Co-authored-by: Antonio <antonio@zed.dev>
Co-authored-by: Antonio Scandurra <me@as-cii.com>
Co-authored-by: Thorsten <thorsten@zed.dev>
Co-authored-by: Max <max@zed.dev>
See https://zed.dev/channel/gpui-536
Fixes https://github.com/zed-industries/zed/issues/9010
Fixes https://github.com/zed-industries/zed/issues/8883
Fixes https://github.com/zed-industries/zed/issues/8640
Fixes https://github.com/zed-industries/zed/issues/8598
Fixes https://github.com/zed-industries/zed/issues/8579
Fixes https://github.com/zed-industries/zed/issues/8363
Fixes https://github.com/zed-industries/zed/issues/8207
### Problem
After transitioning Zed to GPUI 2, we started noticing that interacting
with the mouse on many UI elements would lead to a pretty annoying
flicker. The main issue with the old approach was that hover state was
calculated based on the previous frame. That is, when computing whether
a given element was hovered in the current frame, we would use
information about the same element in the previous frame.
However, inspecting the previous frame tells us very little about what
should be hovered in the current frame, as elements in the current frame
may have changed significantly.
### Solution
This pull request's main contribution is the introduction of a new
`after_layout` phase when redrawing the window. The key idea is that
we'll give every element a chance to register a hitbox (see
`ElementContext::insert_hitbox`) before painting anything. Then, during
the `paint` phase, elements can determine whether they're the topmost
and draw their hover state accordingly.
We are also removing the ability to give an arbitrary z-index to
elements. Instead, we will follow the much simpler painter's algorithm.
That is, an element that gets painted after will be drawn on top of an
element that got painted earlier. Elements can still escape their
current "stacking context" by using the new `ElementContext::defer_draw`
method (see `Overlay` for an example). Elements drawn using this method
will still be logically considered as being children of their original
parent (for keybinding, focus and cache invalidation purposes) but their
layout and paint passes will be deferred until the currently-drawn
element is done.
With these changes we also reworked geometry batching within the
`Scene`. The new approach uses an AABB tree to determine geometry
occlusion, which allows the GPU to render non-overlapping geometry in
parallel.
### Performance
Performance is slightly better than on `main` even though this new
approach is more correct and we're maintaining an extra data structure
(the AABB tree).
Release Notes:
- Fixed a bug that was causing popovers to flicker.
---------
Co-authored-by: Nathan Sobo <nathan@zed.dev>
Co-authored-by: Thorsten <thorsten@zed.dev>
We've also upgraded `Axum` in order to avoid having two versions of that
library in Collab (one due to Tokio-console).
Release Notes:
- N/A
---------
Co-authored-by: Conrad <conrad@zed.dev>
This practice makes it difficult to locate todo!s in my code when I'm
working. Let's take out the bang if we want to keep doing this.
Release Notes:
- N/A
This PR makes it so the background task that reconciles the extensions
database with the blob store only runs on the `collab` service.
This avoids us having multiple of these jobs running at once.
Release Notes:
- N/A
Co-Authored-By: Marshall <marshall@zed.dev>
We broke it by deploying two servers simultaneously.
Release Notes:
- N/A
Co-authored-by: Marshall <marshall@zed.dev>
This PR adds a REST API to the collab server for searching and
downloading extensions. Previously, we had implemented this API in
zed.dev directly, but this implementation is better, because we use the
collab database to store the download counts for extensions.
Release Notes:
- N/A
---------
Co-authored-by: Marshall Bowers <elliott.codes@gmail.com>
Co-authored-by: Marshall <marshall@zed.dev>
Co-authored-by: Conrad <conrad@zed.dev>
After this change we'll be able to push a tag to github to deploy to
collab.
The advantages of this are that there's no longer a separate step to
first
build the image, and then deploy it.
In the future I'd like to make this happen more automatically (maybe as
part of
bump nightly).
Release Notes:
- N/A
- Disallow sharing gitignored files through collab
- Show errors when failing to open files
- Show a warning to followers when view is unshared
/cc @mikaylamaki, let's update this to use your `private_files` config
before merge.
Release Notes:
- Added the ability to prevent sharing private files over collab.
---------
Co-authored-by: Piotr <piotr@zed.dev>
Co-authored-by: Mikayla <mikayla@zed.dev>
