More fixes to the semantic index's chunking (#11376)

This fixes a tricky intermittent issue I was seeing, where failed to
chunk certain files correctly because of the way we reuse Tree-sitter
`Parser` instances across parses.

I've also accounted for leading comments in chunk boundaries, so that
items are grouped with their leading comments whenever possible when
chunking.

Finally, we've changed the `debug project index` action so that it opens
a simple debug view in a pane, instead of printing paths to the console.
This lets you click into a path and see how it was chunked.

Release Notes:

- N/A

---------

Co-authored-by: Marshall <marshall@zed.dev>
This commit is contained in:
Max Brunsfeld 2024-05-03 19:00:18 -07:00 committed by GitHub
parent 335c307b93
commit 6964302d89
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
11 changed files with 532 additions and 171 deletions

View file

@ -3,8 +3,8 @@ use crate::{
all_language_settings, AllLanguageSettingsContent, LanguageSettingsContent,
},
task_context::ContextProvider,
CachedLspAdapter, File, Language, LanguageConfig, LanguageId, LanguageMatcher,
LanguageServerName, LspAdapter, LspAdapterDelegate, PARSER, PLAIN_TEXT,
with_parser, CachedLspAdapter, File, Language, LanguageConfig, LanguageId, LanguageMatcher,
LanguageServerName, LspAdapter, LspAdapterDelegate, PLAIN_TEXT,
};
use anyhow::{anyhow, Context as _, Result};
use collections::{hash_map, HashMap};
@ -668,8 +668,7 @@ impl LanguageRegistry {
.file_stem()
.and_then(OsStr::to_str)
.ok_or_else(|| anyhow!("invalid grammar filename"))?;
anyhow::Ok(PARSER.with(|parser| {
let mut parser = parser.borrow_mut();
anyhow::Ok(with_parser(|parser| {
let mut store = parser.take_wasm_store().unwrap();
let grammar = store.load_language(&grammar_name, &wasm_bytes);
parser.set_wasm_store(store).unwrap();