Add telemetry for supermaven (#11821)

Data migration plan:

- [X] Make a duplicate table of `copilot_events`
    - Name: `inline_completion_events`
    - Omit `suggestion_id` column
- [X-reverted-skipping] In collab, continue to match on copilot_events,
but simply stuff their data into inline_completion_events, to forward it
to the new table
- [skipping] Once collab is deployed, ensure no events are being sent to
copilot_events, migrate `copilot_events` to new table via a transaction
- [skipping] Delete `copilot_events` table

---

- [X] Locally test that copilot events sent from old clients get put
into inline_completions_table
- [X] Locally test that copilot events and supermaven events sent from
new clients get put into inline_completions_table

---

- [X] Why are discard events being spammed?
- A:
8d4315712b/crates/editor/src/editor.rs (L2147)


![scr-20240514-pqmg](https://github.com/zed-industries/zed/assets/19867440/e51e7ae4-21b8-47a2-bfaa-f68fb355e409)

This will throw off the past results for accepted / dismissed that I was
wanting to use to evaluate Supermaven quality, by comparing its rate
with copilot's rate.

I'm not super thrilled with this fix, but I think it'll do. In the
`supermaven_completions_provider`, we check if there's a `completion_id`
before sending either an accepted or discard completion event. I don't
see a similar construct in the `copilot_completions_provider` to
piggyback off of, so I begrudgingly introduced
`should_allow_event_to_send` and had it follow the same pattern that
`completion_id` does. Maybe there's a better way?

---

Adds events to supermaven suggestions. Makes "CopilotEvents" generic ->
"InlineCompletionEvents".

Release Notes:

- N/A
This commit is contained in:
Joseph T. Lyons 2024-05-16 17:18:32 -04:00 committed by GitHub
parent 55f08c0511
commit b6189b05f9
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
9 changed files with 123 additions and 42 deletions

View file

@ -22,6 +22,7 @@ pub struct CopilotCompletionProvider {
pending_cycling_refresh: Task<Result<()>>,
copilot: Model<Copilot>,
telemetry: Option<Arc<Telemetry>>,
should_allow_event_to_send: bool,
}
impl CopilotCompletionProvider {
@ -36,6 +37,7 @@ impl CopilotCompletionProvider {
pending_cycling_refresh: Task::ready(Ok(())),
copilot,
telemetry: None,
should_allow_event_to_send: false,
}
}
@ -59,6 +61,10 @@ impl CopilotCompletionProvider {
}
impl InlineCompletionProvider for CopilotCompletionProvider {
fn name() -> &'static str {
"copilot"
}
fn is_enabled(
&self,
buffer: &Model<Buffer>,
@ -99,6 +105,7 @@ impl InlineCompletionProvider for CopilotCompletionProvider {
this.update(&mut cx, |this, cx| {
if !completions.is_empty() {
this.should_allow_event_to_send = true;
this.cycled = false;
this.pending_cycling_refresh = Task::ready(Ok(()));
this.completions.clear();
@ -187,13 +194,17 @@ impl InlineCompletionProvider for CopilotCompletionProvider {
.update(cx, |copilot, cx| copilot.accept_completion(completion, cx))
.detach_and_log_err(cx);
if let Some(telemetry) = self.telemetry.as_ref() {
telemetry.report_copilot_event(
Some(completion.uuid.clone()),
true,
self.file_extension.clone(),
);
if self.should_allow_event_to_send {
telemetry.report_inline_completion_event(
Self::name().to_string(),
true,
self.file_extension.clone(),
);
}
}
}
self.should_allow_event_to_send = false;
}
fn discard(&mut self, cx: &mut ModelContext<Self>) {
@ -210,9 +221,18 @@ impl InlineCompletionProvider for CopilotCompletionProvider {
copilot.discard_completions(&self.completions, cx)
})
.detach_and_log_err(cx);
if let Some(telemetry) = self.telemetry.as_ref() {
telemetry.report_copilot_event(None, false, self.file_extension.clone());
if self.should_allow_event_to_send {
telemetry.report_inline_completion_event(
Self::name().to_string(),
false,
self.file_extension.clone(),
);
}
}
self.should_allow_event_to_send = false;
}
fn active_completion_text<'a>(