lsp: Implement support for the textDocument/diagnostic command (#19230)

Closes [#13107](https://github.com/zed-industries/zed/issues/13107)

Enabled pull diagnostics by default, for the language servers that
declare support in the corresponding capabilities.

```
"diagnostics": {
    "lsp_pull_diagnostics_debounce_ms": null
}
```
settings can be used to disable the pulling.

Release Notes:

- Added support for the LSP `textDocument/diagnostic` command.

# Brief

This is draft PR that implements the LSP `textDocument/diagnostic`
command. The goal is to receive your feedback and establish further
steps towards fully implementing this command. I tried to re-use
existing method and structures to ensure:

1. The existing functionality works as before
2. There is no interference between the diagnostics sent by a server and
the diagnostics requested by a client.

The current implementation is done via a new LSP command
`GetDocumentDiagnostics` that is sent when a buffer is saved and when a
buffer is edited. There is a new method called `pull_diagnostic` that is
called for such events. It has debounce to ensure we don't spam a server
with commands every time the buffer is edited. Probably, we don't need
the debounce when the buffer is saved.

All in all, the goal is basically to get your feedback and ensure I am
on the right track. Thanks!


## References

1.
https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_pullDiagnostics

## In action

You can clone any Ruby repo since the `ruby-lsp` supports the pull
diagnostics only.

Steps to reproduce:

1. Clone this repo https://github.com/vitallium/stimulus-lsp-error-zed
2. Install Ruby (via `asdf` or `mise).
4. Install Ruby gems via `bundle install`
5. Install Ruby LSP with `gem install ruby-lsp`
6. Check out this PR and build Zed
7. Open any file and start editing to see diagnostics in realtime.



https://github.com/user-attachments/assets/0ef6ec41-e4fa-4539-8f2c-6be0d8be4129

---------

Co-authored-by: Kirill Bulatov <mail4score@gmail.com>
Co-authored-by: Kirill Bulatov <kirill@zed.dev>

crates/project/src/project.rs

@@ -72,9 +72,9 @@ use gpui::{
 };
 use itertools::Itertools;
 use language::{
-    Buffer, BufferEvent, Capability, CodeLabel, CursorShape, Language, LanguageName,
-    LanguageRegistry, PointUtf16, ToOffset, ToPointUtf16, Toolchain, ToolchainList, Transaction,
-    Unclipped, language_settings::InlayHintKind, proto::split_operations,
+    Buffer, BufferEvent, Capability, CodeLabel, CursorShape, DiagnosticSourceKind, Language,
+    LanguageName, LanguageRegistry, PointUtf16, ToOffset, ToPointUtf16, Toolchain, ToolchainList,
+    Transaction, Unclipped, language_settings::InlayHintKind, proto::split_operations,
 };
 use lsp::{
     CodeActionKind, CompletionContext, CompletionItemKind, DocumentHighlightKind, InsertTextMode,
@@ -317,6 +317,7 @@ pub enum Event {
     SnippetEdit(BufferId, Vec<(lsp::Range, Snippet)>),
     ExpandedAllForEntry(WorktreeId, ProjectEntryId),
     AgentLocationChanged,
+    RefreshDocumentsDiagnostics,
 }
 
 pub struct AgentLocationChanged;
@@ -861,6 +862,34 @@ pub const DEFAULT_COMPLETION_CONTEXT: CompletionContext = CompletionContext {
     trigger_character: None,
 };
 
+/// An LSP diagnostics associated with a certain language server.
+#[derive(Clone, Debug, Default)]
+pub enum LspPullDiagnostics {
+    #[default]
+    Default,
+    Response {
+        /// The id of the language server that produced diagnostics.
+        server_id: LanguageServerId,
+        /// URI of the resource,
+        uri: lsp::Url,
+        /// The diagnostics produced by this language server.
+        diagnostics: PulledDiagnostics,
+    },
+}
+
+#[derive(Clone, Debug)]
+pub enum PulledDiagnostics {
+    Unchanged {
+        /// An ID the current pulled batch for this file.
+        /// If given, can be used to query workspace diagnostics partially.
+        result_id: String,
+    },
+    Changed {
+        result_id: Option<String>,
+        diagnostics: Vec<lsp::Diagnostic>,
+    },
+}
+
 impl Project {
     pub fn init_settings(cx: &mut App) {
         WorktreeSettings::register(cx);
@@ -2785,6 +2814,9 @@ impl Project {
             }
             LspStoreEvent::RefreshInlayHints => cx.emit(Event::RefreshInlayHints),
             LspStoreEvent::RefreshCodeLens => cx.emit(Event::RefreshCodeLens),
+            LspStoreEvent::RefreshDocumentsDiagnostics => {
+                cx.emit(Event::RefreshDocumentsDiagnostics)
+            }
             LspStoreEvent::LanguageServerPrompt(prompt) => {
                 cx.emit(Event::LanguageServerPrompt(prompt.clone()))
             }
@@ -3686,6 +3718,35 @@ impl Project {
         })
     }
 
+    pub fn document_diagnostics(
+        &mut self,
+        buffer_handle: Entity<Buffer>,
+        cx: &mut Context<Self>,
+    ) -> Task<Result<Vec<LspPullDiagnostics>>> {
+        self.lsp_store.update(cx, |lsp_store, cx| {
+            lsp_store.pull_diagnostics(buffer_handle, cx)
+        })
+    }
+
+    pub fn update_diagnostics(
+        &mut self,
+        language_server_id: LanguageServerId,
+        source_kind: DiagnosticSourceKind,
+        params: lsp::PublishDiagnosticsParams,
+        disk_based_sources: &[String],
+        cx: &mut Context<Self>,
+    ) -> Result<(), anyhow::Error> {
+        self.lsp_store.update(cx, |lsp_store, cx| {
+            lsp_store.update_diagnostics(
+                language_server_id,
+                params,
+                source_kind,
+                disk_based_sources,
+                cx,
+            )
+        })
+    }
+
     pub fn search(&mut self, query: SearchQuery, cx: &mut Context<Self>) -> Receiver<SearchResult> {
         let (result_tx, result_rx) = smol::channel::unbounded();