11 commits

Author	SHA1	Message	Date
KCaverly	76ce52df4e	move queuing to embedding_queue functionality and update embedding provider to include trait items for max tokens per batch" ... Co-authored-by: Max <max@zed.dev>	2023-08-30 16:01:28 -04:00
KCaverly	9781047156	move truncation to parsing step leveraging the EmbeddingProvider trait	2023-08-30 12:13:26 -04:00
KCaverly	e377ada1a9	added token count to documents during parsing	2023-08-30 11:05:46 -04:00
KCaverly	3d89cd10a4	added sha1 encoding for each document	2023-08-21 16:35:57 +02:00
KCaverly	e221f23018	add support for markdown files to semantic search	2023-08-01 10:30:34 -04:00
KCaverly	9a50b43eaa	add templating languages html, erb, heex, svelte as entire parseable file types	2023-07-31 21:03:02 -04:00
KCaverly	599f674827	add php support for semantic search	2023-07-31 16:36:09 -04:00
KCaverly	89edb3d1b5	fix templating bug for parseable entire files	2023-07-31 11:41:18 -04:00
KCaverly	efe973ebe2	add embedding query for json with nested arrays and strings ... Co-authored-by: maxbrunsfeld <max@zed.dev>	2023-07-19 16:52:44 -04:00
KCaverly	9809ec3d70	update treesitter parsing to accomodate for collapsed nested functions ... Co-authored-by: maxbrunsfeld <max@zed.dev>	2023-07-19 15:47:05 -04:00
KCaverly	8b42f5b1b3	rename vector_store crate to semantic_index	2023-07-17 17:06:10 -04:00

Renamed from crates/vector_store/src/parsing.rs (Browse further)