6ff69faf37
This PR adds support for sending telemetry events to AWS Kinesis. In our AWS account we now have three new things: * The [Kinesis data stream](https://us-east-1.console.aws.amazon.com/kinesis/home?region=us-east-1#/streams/details/zed-telemetry/monitoring) that we will actually write to. * A [Firehose for Axiom](https://us-east-1.console.aws.amazon.com/firehose/home?region=us-east-1#/details/telemetry-to-axiom/monitoring) that sends events from that stream to Axiom for ad-hoc queries over recent data. * A [Firehose for Snowflake](https://us-east-1.console.aws.amazon.com/firehose/home?region=us-east-1#/details/telemetry-to-snowflake/monitoring) that sends events from that stream to Snowflake for long-term retention. This Firehose also backs up data into an S3 bucket in case we want to change how the system works in the future. In a follow-up PR, we'll add support for ad-hoc telemetry events; and slowly move away from the current Clickhouse defined schemas; though we won't move off click house until we have what we need in Snowflake. Co-Authored-By: Nathan <nathan@zed.dev> Release Notes: - N/A |
||
|---|---|---|
| .. | ||
| k8s | ||
| migrations | ||
| migrations.sqlite | ||
| migrations_llm | ||
| seed | ||
| src | ||
| .env.toml | ||
| Cargo.toml | ||
| LICENSE-AGPL | ||
| postgrest_app.conf | ||
| postgrest_llm.conf | ||
| README.md | ||
| seed.default.json | ||
Zed Server
This crate is what we run at https://collab.zed.dev.
It contains our back-end logic for collaboration, to which we connect from the Zed client via a websocket after authenticating via https://zed.dev, which is a separate repo running on Vercel.
Local Development
Database setup
Before you can run the collab server locally, you'll need to set up a zed Postgres database.
script/bootstrap
This script will set up the zed Postgres database, and populate it with some users. It requires internet access, because it fetches some users from the GitHub API.
The script will create several admin users, who you'll sign in as by default when developing locally. The GitHub logins for the default users are specified in the seed.default.json file.
To use a different set of admin users, create crates/collab/seed.json.
{
"admins": ["yourgithubhere"],
"channels": ["zed"]
}
Testing collaborative features locally
In one terminal, run Zed's collaboration server and the livekit dev server:
foreman start
In a second terminal, run two or more instances of Zed.
script/zed-local -2
This script starts one to four instances of Zed, depending on the -2, -3 or -4 flags. Each instance will be connected to the local collab server, signed in as a different user from seed.json or seed.default.json.
Deployment
We run two instances of collab:
- Staging (https://staging-collab.zed.dev)
- Production (https://collab.zed.dev)
Both of these run on the Kubernetes cluster hosted in Digital Ocean.
Deployment is triggered by pushing to the collab-staging (or collab-production) tag in Github. The best way to do this is:
./script/deploy-collab staging./script/deploy-collab production
You can tell what is currently deployed with ./script/what-is-deployed.
Database Migrations
To create a new migration:
./script/create-migration <name>
Migrations are run automatically on service start, so run foreman start again. The service will crash if the migrations fail.
When you create a new migration, you also need to update the SQLite schema that is used for testing.