Yehowshua/ZIm
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
ZIm/crates
History
Oleksiy Syvokon c874f1fa9d
agent: Migrate thread storage to SQLite with zstd compression (#31741) 
Previously, LMDB was used for storing threads, but it consumed excessive
disk space and was capped at 1GB.

This change migrates thread storage to an SQLite database. Thread JSON
objects are now compressed using zstd.

I considered training a custom zstd dictionary and storing it in a
separate table. However, the additional complexity outweighed the modest
space savings (up to 20%). I ended up using the default dictionary
stored with data.

Threads can be exported relatively easily from outside the application:

```
$ sqlite3 threads.db "SELECT hex(data) FROM threads LIMIT 5;" |
    xxd -r -p |
    zstd -d |
    fx
```

Benchmarks:
- Original heed database: 200MB
- Sqlite uncompressed: 51MB
- sqlite compressed (this PR): 4.0MB
- sqlite compressed with a trained dictionary: 3.8MB


Release Notes:

- Migrated thread storage to SQLite with compression
2025-06-02 17:01:34 +03:00
..
activity_indicator debugger: Add spinners while session is starting up (#31548) 2025-05-29 01:58:40 +00:00
agent agent: Migrate thread storage to SQLite with zstd compression (#31741) 2025-06-02 17:01:34 +03:00
agent_settings Fix existing CompletionMode deserialization (#31790) 2025-05-30 19:06:08 -03:00
anthropic anthropic: Fix Claude 4 model display names to match official order (#31218) 2025-05-22 21:00:54 +00:00
askpass askpass: Workaround rust lang 69343 (#30774) 2025-05-16 05:04:36 -04:00
assets Use anyhow more idiomatically (#31052) 2025-05-20 23:06:07 +00:00
assistant_context_editor assistant_context_editor: Fix copy paste regression (#31882) 2025-06-02 11:52:47 +02:00
assistant_slash_command chore: Make terminal_view own the TerminalSlashCommand (#31070) 2025-05-21 09:27:54 +00:00
assistant_slash_commands zlog: Replace usages of env_logger in tests with zlog (#31436) 2025-05-26 11:48:50 -04:00
assistant_tool agent: Rejecting agent changes shouldn't discard user edits (#31617) 2025-05-28 18:44:49 +00:00
assistant_tools assistant_tools: Reduce allocations (#30776) 2025-05-30 08:28:22 -07:00
audio agent: Add sound notification when done generating (#31472) 2025-05-26 21:20:41 -03:00
auto_update Show version info when downloading and installing updates (#31568) 2025-05-28 11:51:21 -04:00
auto_update_helper Use anyhow more idiomatically (#31052) 2025-05-20 23:06:07 +00:00
auto_update_ui markdown_preview: Fix release notes title being overridden (#31703) 2025-05-30 08:29:52 +00:00
aws_http_client
bedrock Update default models to newer versions (#31415) 2025-05-27 10:54:42 +02:00
breadcrumbs breadcrumbs: Stylize filename in breadcrumbs when tab-bar is off and file is dirty (#30507) 2025-05-30 08:32:54 -07:00
buffer_diff zlog: Replace usages of env_logger in tests with zlog (#31436) 2025-05-26 11:48:50 -04:00
call Use read-only access methods for read-only entity operations (#31479) 2025-05-26 23:04:31 -04:00
channel Use read-only access methods for read-only entity operations (#31479) 2025-05-26 23:04:31 -04:00
cli Use anyhow more idiomatically (#31052) 2025-05-20 23:06:07 +00:00
client Use read-only access methods for read-only entity operations (#31479) 2025-05-26 23:04:31 -04:00
clock Add the ability to follow the agent as it makes edits (#29839) 2025-05-04 08:28:39 +00:00
collab Refactor item-closing actions (#31838) 2025-05-31 19:38:32 -04:00
collab_ui Use read-only access methods for read-only entity operations (#31479) 2025-05-26 23:04:31 -04:00
collections
command_palette Use read-only access methods for read-only entity operations (#31479) 2025-05-26 23:04:31 -04:00
command_palette_hooks
component component: Replace linkme with inventory (#30705) 2025-05-14 23:29:11 +02:00
context_server context_server: Fix casing of mimeType in tool responses (#30703) 2025-05-26 10:29:45 +02:00
copilot Revert "copilot: Fix vision request detection for follow-up messages" (#31776) 2025-05-30 21:28:31 +02:00
credentials_provider
dap Do not error on debugger server connection close (#31795) 2025-05-30 22:37:40 +00:00
dap_adapters debugger: Relax implementation of validate_config to not run validation (#31785) 2025-05-30 23:08:41 +02:00
db Add end of service notifications (#30982) 2025-05-20 00:20:00 +00:00
debug_adapter_extension debugger beta: Fix dap_schema for DAP extensions (#31173) 2025-05-22 07:24:46 -04:00
debugger_tools Rename debug: commands to dev: (#30675) 2025-05-14 11:15:27 +02:00
debugger_ui debugger: Relax implementation of validate_config to not run validation (#31785) 2025-05-30 23:08:41 +02:00
deepseek Use anyhow more idiomatically (#31052) 2025-05-20 23:06:07 +00:00
diagnostics Use read-only access methods for read-only entity operations (#31479) 2025-05-26 23:04:31 -04:00
docs_preprocessor docs_preprocessor: Ensure keybind is found for actions with arguments (#27224) 2025-05-27 16:56:03 +00:00
editor editor: Fix columnar selection incorrectly uses cursor to start selection instead of mouse position (#31888) 2025-06-02 16:37:36 +05:30
eval Pass up intent with completion requests (#31710) 2025-05-29 20:43:12 +00:00
extension extension: Update to wasm32-wasip2 target (#30953) 2025-05-26 16:06:38 +02:00
extension_api extension: Update to wasm32-wasip2 target (#30953) 2025-05-26 16:06:38 +02:00
extension_cli Use anyhow more idiomatically (#31052) 2025-05-20 23:06:07 +00:00
extension_host zlog: Replace usages of env_logger in tests with zlog (#31436) 2025-05-26 11:48:50 -04:00
extensions_ui Remove block_mouse_down in favor of stop_mouse_events_except_scroll (#30401) 2025-05-29 22:07:34 +00:00
feature_flags Add a picker for jj bookmark list (#30883) 2025-05-17 16:42:45 +00:00
feedback Use shortened SHA when displaying version to install (#31281) 2025-05-23 14:53:53 +00:00
file_finder Improve the file finder picker footer design (#31777) 2025-06-02 09:54:15 -03:00
file_icons
fs Use anyhow more idiomatically (#31052) 2025-05-20 23:06:07 +00:00
fsevent
fuzzy Fix out-of-bounds panic in fuzzy matcher with Unicode/multibyte characters (#30546) 2025-05-12 14:43:14 +00:00
git agent: Don't track large and common binary files (#31352) 2025-05-26 11:31:25 +00:00
git_hosting_providers Use anyhow more idiomatically (#31052) 2025-05-20 23:06:07 +00:00
git_ui Pass up intent with completion requests (#31710) 2025-05-29 20:43:12 +00:00
go_to_line editor: Add minimap (#26893) 2025-05-07 23:11:09 +03:00
google_ai Update default models to newer versions (#31415) 2025-05-27 10:54:42 +02:00
gpui gpui: Implement window_handle and display_handle for wayland platform (#28152) 2025-05-30 15:45:03 -07:00
gpui_macros Add a live Rust style editor to inspector to edit a sequence of no-argument style modifiers (#31443) 2025-05-26 17:43:57 +00:00
gpui_tokio
html_to_markdown
http_client Use anyhow more idiomatically (#31052) 2025-05-20 23:06:07 +00:00
http_client_tls
icons Harmonize quick action icons (#31784) 2025-05-30 18:18:23 -03:00
image_viewer Use anyhow more idiomatically (#31052) 2025-05-20 23:06:07 +00:00
indexed_docs Use anyhow more idiomatically (#31052) 2025-05-20 23:06:07 +00:00
inline_completion Use anyhow more idiomatically (#31052) 2025-05-20 23:06:07 +00:00
inline_completion_button inline_completion_button: Fix links to account page (#31558) 2025-05-27 21:52:42 +00:00
inspector_ui Add a live Rust style editor to inspector to edit a sequence of no-argument style modifiers (#31443) 2025-05-26 17:43:57 +00:00
install_cli Use anyhow more idiomatically (#31052) 2025-05-20 23:06:07 +00:00
jj Add a picker for jj bookmark list (#30883) 2025-05-17 16:42:45 +00:00
jj_ui Add a picker for jj bookmark list (#30883) 2025-05-17 16:42:45 +00:00
journal VSCode Settings import (#29018) 2025-04-23 20:54:09 +00:00
language Improve TypeScript task detection (#31711) 2025-05-29 20:51:20 +00:00
language_extension debugger/extensions: Revert changes to extension store related to language config (#30225) 2025-05-08 14:01:39 +02:00
language_model Pass up intent with completion requests (#31710) 2025-05-29 20:43:12 +00:00
language_models agent: Fix bug with double-counting tokens in Gemini (#31885) 2025-06-02 10:18:44 +00:00
language_selector Use anyhow more idiomatically (#31052) 2025-05-20 23:06:07 +00:00
language_tools language_tools: Increase available space for language server logs (#30742) 2025-05-27 00:57:45 +03:00
languages python: Respect picked toolchain (when it's not at the root) when running tests (#31150) 2025-06-02 15:29:34 +02:00
livekit_api Use anyhow more idiomatically (#31052) 2025-05-20 23:06:07 +00:00
livekit_client Use anyhow more idiomatically (#31052) 2025-05-20 23:06:07 +00:00
lmstudio language_models: Add support for tool use to LM Studio provider (#30589) 2025-05-26 13:54:17 +02:00
lsp zlog: Replace usages of env_logger in tests with zlog (#31436) 2025-05-26 11:48:50 -04:00
markdown gpui: Support hitbox blocking mouse interaction except scrolling (#31712) 2025-05-29 21:41:15 +00:00
markdown_preview markdown_preview: Fix release notes title being overridden (#31703) 2025-05-30 08:29:52 +00:00
media Use anyhow more idiomatically (#31052) 2025-05-20 23:06:07 +00:00
menu agent: Add new panel navigation dropdown (#29539) 2025-04-29 21:58:45 -03:00
migrator agent: Add Burn Mode setting migrator (#31718) 2025-05-30 08:10:12 -03:00
mistral mistral: Add DevstralSmallLatest model to Mistral and Ollama (#31099) 2025-05-22 14:22:35 -04:00
multi_buffer Fix editor rendering slowness with large folds (#31569) 2025-05-28 23:05:06 +00:00
node_runtime Fix unzipping clangd and codelldb on Windows (#31080) 2025-05-21 21:17:14 +03:00
notifications component: Replace linkme with inventory (#30705) 2025-05-14 23:29:11 +02:00
ollama ollama: Change default context size to 4096 (#31682) 2025-05-30 16:12:39 -04:00
open_ai open_ai: Fix parallel tools issue (#30467) 2025-05-26 11:46:35 +00:00
outline Use read-only access methods for read-only entity operations (#31479) 2025-05-26 23:04:31 -04:00
outline_panel Use read-only access methods for read-only entity operations (#31479) 2025-05-26 23:04:31 -04:00
panel
paths Cursor settings import (#31424) 2025-05-27 14:14:25 -04:00
picker Allow to reuse PickerPopoverMenu outside of the model selector (#31684) 2025-05-29 12:55:47 +00:00
prettier Use anyhow more idiomatically (#31052) 2025-05-20 23:06:07 +00:00
project debugger: Add support for go tests (#31772) 2025-05-30 22:18:32 +02:00
project_panel Use read-only access methods for read-only entity operations (#31479) 2025-05-26 23:04:31 -04:00
project_symbols Use read-only access methods for read-only entity operations (#31479) 2025-05-26 23:04:31 -04:00
prompt_store Use anyhow more idiomatically (#31052) 2025-05-20 23:06:07 +00:00
proto Allow LSP adapters to decide, which diagnostics to underline (#31450) 2025-05-26 22:19:02 +03:00
recent_projects recent_projects: Move SSH server entry to initialize once instead of every render (#31650) 2025-05-29 09:24:39 +05:30
refineable Add a live Rust style editor to inspector to edit a sequence of no-argument style modifiers (#31443) 2025-05-26 17:43:57 +00:00
release_channel Use shortened SHA when displaying version to install (#31281) 2025-05-23 14:53:53 +00:00
remote Make it possible to use cargo-zigbuild for ZED_BUILD_REMOTE_SERVER (#31467) 2025-05-27 16:56:27 +00:00
remote_server Remove redundant words in comments (#31512) 2025-05-27 23:01:31 +00:00
repl Remove block_mouse_down in favor of stop_mouse_events_except_scroll (#30401) 2025-05-29 22:07:34 +00:00
reqwest_client Use anyhow more idiomatically (#31052) 2025-05-20 23:06:07 +00:00
rich_text
rope zlog: Replace usages of env_logger in tests with zlog (#31436) 2025-05-26 11:48:50 -04:00
rpc zlog: Replace usages of env_logger in tests with zlog (#31436) 2025-05-26 11:48:50 -04:00
rules_library Pass up intent with completion requests (#31710) 2025-05-29 20:43:12 +00:00
schema_generator
search Use read-only access methods for read-only entity operations (#31479) 2025-05-26 23:04:31 -04:00
semantic_index Pass up intent with completion requests (#31710) 2025-05-29 20:43:12 +00:00
semantic_version Use anyhow more idiomatically (#31052) 2025-05-20 23:06:07 +00:00
session Avoid unnecessary DB writes (#29417) 2025-04-25 17:41:49 +03:00
settings Cursor settings import (#31424) 2025-05-27 14:14:25 -04:00
settings_ui Cursor settings import (#31424) 2025-05-27 14:14:25 -04:00
snippet Use anyhow more idiomatically (#31052) 2025-05-20 23:06:07 +00:00
snippet_provider Use read-only access methods for read-only entity operations (#31479) 2025-05-26 23:04:31 -04:00
snippets_ui snippets: Add icons and file names to snippet scope selector (#30212) 2025-05-26 13:44:09 +00:00
sqlez Use anyhow more idiomatically (#31052) 2025-05-20 23:06:07 +00:00
sqlez_macros
story gpui: Add a standard text example (#30747) 2025-05-16 17:35:44 +02:00
storybook Use anyhow more idiomatically (#31052) 2025-05-20 23:06:07 +00:00
streaming_diff Introduce a new StreamingEditFileTool (#29733) 2025-05-01 17:37:43 +02:00
sum_tree zlog: Replace usages of env_logger in tests with zlog (#31436) 2025-05-26 11:48:50 -04:00
supermaven
supermaven_api Keep file permissions when extracting zip archives on Unix (#31304) 2025-05-23 20:45:32 +00:00
tab_switcher tab_switcher: Add placeholder text (#31697) 2025-05-29 16:09:07 +00:00
task Revert "debugger beta: Fix bug where debug Rust main running action f… (#31743) 2025-05-30 14:32:59 +02:00
tasks_ui Use both language and LSP icons for LSP tasks (#31773) 2025-05-30 19:28:56 +00:00
telemetry
telemetry_events
terminal terminal: Consume event during processing (#30869) 2025-05-23 14:28:53 +03:00
terminal_view Fix slowness in Terminal when vi-mode is enabled (#31824) 2025-05-31 20:02:56 +03:00
text Show progress as the agent locates which range it needs to edit (#31582) 2025-05-28 12:32:54 +00:00
theme theme: Add colors for minimap thumb and border (#30785) 2025-05-26 18:23:41 +00:00
theme_extension
theme_importer theme: Add colors for minimap thumb and border (#30785) 2025-05-26 18:23:41 +00:00
theme_selector Icon theme selector: Don't select last list item when fuzzy searching (#29560) 2025-05-30 20:37:38 +00:00
time_format agent: Add date separators to Thread History (#29961) 2025-05-06 10:18:48 +00:00
title_bar Support macOS Sequoia titlebar double-click action (#30468) 2025-05-30 17:13:50 +00:00
toolchain_selector Use read-only access methods for read-only entity operations (#31479) 2025-05-26 23:04:31 -04:00
ui gpui: Support hitbox blocking mouse interaction except scrolling (#31712) 2025-05-29 21:41:15 +00:00
ui_input component: Replace linkme with inventory (#30705) 2025-05-14 23:29:11 +02:00
ui_macros Update syn crate from 1.0.109 to 2.0.101 (#31301) 2025-05-23 19:31:25 +00:00
ui_prompt gpui: Improve window.prompt to support ESC with non-English cancel text on macOS (#29538) 2025-05-30 15:26:27 +00:00
util Revert "Fix text wrapping in commit message editors (#31030)" (#31587) 2025-05-28 10:16:34 -04:00
util_macros Fix license symlinks (#29758) 2025-05-01 19:24:14 +00:00
vim vim: Fix crash when using ‘ge’ motion on multibyte character (#31566) 2025-05-28 06:30:51 +05:30
vim_mode_setting VSCode Settings import (#29018) 2025-04-23 20:54:09 +00:00
web_search
web_search_providers Use anyhow more idiomatically (#31052) 2025-05-20 23:06:07 +00:00
welcome Cursor keymap (#31702) 2025-05-29 15:20:58 -04:00
workspace Notify when pinning a tab even if the tab isn't moved (#31880) 2025-06-02 06:58:57 +00:00
worktree Use read-only access methods for read-only entity operations (#31479) 2025-05-26 23:04:31 -04:00
zed Ensure item-closing actions do not panic when no items are present (#31845) 2025-06-01 03:31:38 +00:00
zed_actions debugger: Add keyboard navigation for breakpoint list (#31221) 2025-05-26 19:40:07 +00:00
zeta zlog: Replace usages of env_logger in tests with zlog (#31436) 2025-05-26 11:48:50 -04:00
zlog zlog: Check crate name against filters if scope empty (#31892) 2025-06-02 13:22:32 +00:00
zlog_settings VSCode Settings import (#29018) 2025-04-23 20:54:09 +00:00