This needs scrutinized. Detailed breakdown of what events I kept and
threw out here:
https://zed.dev/channel/app-events-17178/notes
I also removed a few fake events and tossed out json properties that
were being inserted for things we don't have logic to track. See PR
review comments below.
I think the only bad data we have are that we were identifying all node,
pnpm, and yarn projects as 'node' in the `project_type` property, so a
few days of lost data there.
Release Notes:
- N/A
This will allow us to use the events table directly in Amplitude, which
lets us use the newer event ingestion flow that detects changes to the
table. Otherwise we'll need a transformation.
I think Amplitude's API is probably a pretty good example to follow for
the raw event schema, even if we don't end up using their product. They
also recommend a "Noun Verbed" format for naming events, so I think we
should go with this. This will help us be consistent and encourage the
author of events to think more clearly about what event they're
reporting.
cc @ConradIrwin
Release Notes:
- N/A
This PR updates the checkout flow to include the `?checkout_complete=1`
query parameter after successfully checking out.
We'll use this on the account page to adapt the UI accordingly.
Release Notes:
- N/A
This PR adds some safeguards to ensure that users do not end up with
multiple active billing subscriptions.
We now do the following:
1. When initiating a checkout, we first make sure the user does not
already have an active subscription.
2. When creating subscriptions in response to Stripe events, we ensure
that we don't already have an active subscription.
Release Notes:
- N/A
This fixes a bug that was causing most users to be unable to use the
LLMs via Zed. It was caused by not using pagination and, instead, always
querying the very first page of stripe events.
Note that we're also allowing processing events generated in the last 24
hours (before, this was only 1 hour). I did this so that we can process
the backlog of events that the aforementioned bug was skipping.
Release Notes:
- N/A
This PR makes it so collab will trigger a refresh for a user's LLM token
whenever their subscription changes.
This allows us to proactively push down changes to their subscription.
In order to facilitate this, the Stripe event processing has been moved
from the `api` service to the `collab` service in order to access the
RPC server.
Release Notes:
- N/A
This PR makes the `StripeBilling` object long-lived so that we can make
better use of the cached data on it.
We now hold it on the `AppState` and spawn a background task to
initialize the cache on startup.
Release Notes:
- N/A
Co-authored-by: Richard <richard@zed.dev>
This PR adds usage-based billing for LLM interactions in the Assistant.
Release Notes:
- N/A
---------
Co-authored-by: Antonio Scandurra <me@as-cii.com>
Co-authored-by: Antonio <antonio@zed.dev>
Co-authored-by: Richard <richard@zed.dev>
Co-authored-by: Richard Feldman <oss@rtfeldman.com>
This PR fixes some issues with our previous approach to synching LLM
usage over to Stripe.
We now have a separate LLM access price in Stripe that is a marker price
to allow us to create the initial subscription with that as its
subscription item
We then dynamically set the LLM usage price during the reconciliation
sync based on the usage for the current month.
Release Notes:
- N/A
---------
Co-authored-by: Antonio <antonio@zed.dev>
Co-authored-by: Richard <richard@zed.dev>
This PR adds a new `billing_preferences` table.
Right now there is a single preference: the maximum monthly spend for
LLM usage.
Release Notes:
- N/A
---------
Co-authored-by: Richard <richard@zed.dev>
This PR renames the `MONTHLY_SPENDING_LIMIT` constant to
`FREE_TIER_MONTHLY_SPENDING_LIMIT` to clarify it.
This will help distinguish it from the user's specified limit on their
paid monthly spending.
Release Notes:
- N/A
This PR adds a new `Cents` type that can be used to represent a monetary
value in cents.
This cuts down on the primitive obsession we were using when dealing
with money in the billing code.
Release Notes:
- N/A
We have a lot of data in Clickhouse. This column was used when migrating
the events dataset between analytics databases and has no purpose today.
Naive maths: 257,170,993 editor event rows * 1 byte per boolean =
257,170,993 bytes, or ~0.24 GB
I'll drop the column after deploying a new collab.
Going forward, I'd like to remove more data that we never touch, to try
to keep things more focused. We should discuss some TTL at some point.
Release Notes:
- N/A
This PR reworks our existing billing code in preparation for charging
based on LLM usage.
We aren't yet exercising the new billing-related code outside of
development.
There are some noteworthy changes for our existing LLM usage tracking:
- A new `monthly_usages` table has been added for tracking usage
per-user, per-model, per-month
- The per-month usage measures have been removed, in favor of the
`monthly_usages` table
- All of the per-month metrics in the Clickhouse rows have been changed
from a rolling 30-day window to a calendar month
Release Notes:
- N/A
---------
Co-authored-by: Antonio Scandurra <me@as-cii.com>
Co-authored-by: Richard <richard@zed.dev>
Co-authored-by: Max <max@zed.dev>
This PR reverts the changes to the Clickhouse event rows that were
included in https://github.com/zed-industries/zed/pull/18414.
The changes don't seem to be correct, as they make the row structs
differ from the underlying table schema.
Release Notes:
- N/A
REplace isahc with ureq everywhere gpui is used.
This should allow us to make http requests without libssl; and avoid a
long-tail of panics caused by ishac.
Release Notes:
- (potentially breaking change) updated our http client
---------
Co-authored-by: Mikayla <mikayla@zed.dev>
I noticed a few places where we were storing `&'static str`s in
`static`s instead of `const`s.
This PR updates them to use `const`.
Release Notes:
- N/A
This PR adds `system_id` to telemetry, which is contained within a new
`global` database (accessible by any release channel of Zed on a single
system). This will help us get a more accurate understanding of user
count, instead of relying on `installationd_id`, which is different per
release channel. This doesn't solve the problem of a user with multiple
machines, but it gets us closer.
Release Notes:
- N/A
This PR fixes the writing of LLM rate limit events to Clickhouse.
We had a table in the table name: `llm_rate_limits` instead of
`llm_rate_limit_events`.
I also extracted a helper function to write to Clickhouse so we can use
it anywhere we need to.
Release Notes:
- N/A
This PR restricts usage of the LLM service to accounts older than 30
days.
We now store the GitHub user's `created_at` timestamp to check the
GitHub account age. If this is not set—which it won't be for existing
users—then we use the `created_at` timestamp in the Zed database.
Release Notes:
- N/A
---------
Co-authored-by: Max <max@zed.dev>
This PR introduces a separate backend service for making LLM calls.
It exposes an HTTP interface that can be called by Zed clients. To call
these endpoints, the client must provide a `Bearer` token. These tokens
are issued/refreshed by the collab service over RPC.
We're adding this in a backwards-compatible way. Right now the access
tokens can only be minted for Zed staff, and calling this separate LLM
service is behind the `llm-service` feature flag (which is not
automatically enabled for Zed staff).
Release Notes:
- N/A
---------
Co-authored-by: Marshall <marshall@zed.dev>
Co-authored-by: Marshall Bowers <elliott.codes@gmail.com>
