Search & Vector Stores - #107
Conversation
Co-authored-by: Copilot <copilot@github.com> Signed-off-by: Mike Nguyen <hey@mike.ee>
Signed-off-by: Mike Nguyen <hey@mike.ee>
|
|
||
| This is a proposal surrounding the implementation of two new specialised state stores without bringing into | ||
| scope the deep integrations planned on implementation of these building blocks. Whilst the two building | ||
| blocks share almost the same envelope, the method of requesting the records can vary significantly and should |
There was a problem hiding this comment.
I'm wondering if it'd make sense to ship a single building block with both of the APIs combined into one. With a single component, we get a unified API for index/upsert/delete documents, and different endpoints for fetching, one for lexical search and one for vector search.
This would be beneficial if a user needs to use both APIs at the same time.
There was a problem hiding this comment.
Unless there's some scenario you can think of where a query to a provider might do some sort of join operation between lexical and vector searches (e.g. we can offload that heavy lifting to the provider - we absolutely should not do any manipulation of this data in the runtime), I would vote they remain separate.
This obviates the need for another (and differing) capabilities functionality at the building block level, it keeps the APIs for each building block simple and focused and it potentially improves the rate at which provider implementations can be developed and added (e.g. they need only implement the API associated with one block at a time, not have to implement a broader set just for v1.0).
Especially if the results from either are instead used alongside local logic in the app, that's the domain of the SDK to offer a usage smooth experience where it's the domain of the runtime to establish connectivity, issue commands and perform data transfer. I think runtime's role is better suited to small, lean building block APIs going forward.
There was a problem hiding this comment.
I was considering this too - I can make it clearer in the proposal but I share the same sentiment as Whit.
The two building blocks can be objectively very different. I feel like it may be running the risk of becoming a feature flag mess/very confusing for implementation where providers have differing feature sets that fall distinctly within the lexical or vector search buckets.
We should focus on having distinct building blocks, if you need to use both APIs at the same time then you'd just need to register another component type with the same provider.
There was a problem hiding this comment.
One thing I'd love to see explicitly spelled out is the story for someone who wants to use both search & vector against the same provider. From ^ it sounds like the answer is to register 2 components pointing to the same server.
When that happens, is the underlying data shared - so an item indexed for search and its vector can be tied together by a common ID or are they independent? I think that matters with AI use cases where you might want to store something once and both search it by keyword && find it by similarity.
And is there a scenario where a single query should do both at once - keyword & similarity combined by the provider itself?
There was a problem hiding this comment.
Rather than trying to design a feature like @cicoyle is describing as a one-off feature for this, I'd much rather take that idea back to the drawing board and figure out if the idea is applicable across more components and design that as a separate runtime feature than necessarily jam in something that potentially complex as a one-off here. Seems like that would dramatically increase the complexity of what's otherwise a pretty straightforward proposal here.
| string store_name = 1; | ||
| string index = 2; | ||
| repeated SearchDocument documents = 3; | ||
| map<string, string> metadata = 10; |
There was a problem hiding this comment.
| map<string, string> metadata = 10; | |
| map<string, string> metadata = 4; |
| string store_name = 1; | ||
| string index = 2; | ||
| repeated string ids = 3; | ||
| map<string, string> metadata = 10; |
There was a problem hiding this comment.
| map<string, string> metadata = 10; | |
| map<string, string> metadata = 4; |
| string store_name = 1; | ||
| string collection = 2; | ||
| repeated VectorRecord records = 3; | ||
| map<string, string> metadata = 10; |
There was a problem hiding this comment.
| map<string, string> metadata = 10; | |
| map<string, string> metadata = 4; |
| string store_name = 1; | ||
| string collection = 2; | ||
| repeated string ids = 3; | ||
| map<string, string> metadata = 10; |
There was a problem hiding this comment.
| map<string, string> metadata = 10; | |
| map<string, string> metadata = 4; |
| string collection = 2; | ||
| repeated string ids = 3; | ||
| bool include_values = 4; | ||
| map<string, string> metadata = 10; |
There was a problem hiding this comment.
| map<string, string> metadata = 10; | |
| map<string, string> metadata = 4; |
| } | ||
|
|
||
| message BatchQueryVectorsRequestAlpha1 { | ||
| string store_name = 1; |
There was a problem hiding this comment.
I see store_name && collection here and inside QueryVectorsRequestAlpha1 - shall we trim the inner msg?
| The current implementations of state stores do not accomodate the necessity to access data not only by | ||
| key-value but an extended querying layer. New 'specialised' building blocks facilitate rapid development. | ||
|
|
||
| There is a specific need from AI workloads to not only query by key but also by 'relevant' records returned |
There was a problem hiding this comment.
Can you add a Use Cases section? I believe it would help ground the separate vs combined API decision - might be good to note under an Alternatives Considered section or similar. There are RAG/agent retrieval patterns where lexical & vector results get fused together. I believe its a first class AI usecase that should be considered, rather than it being an edge case. Its worth deciding if we should have the APIs separate and/or have a hybrid, even if the hybrid API is out of scope for now - it should be thought thru ahead of implementation to prevent breaking changes later on.
There was a problem hiding this comment.
One of the key ideas of my proposal to split out into discrete building blocks is to explicitly pivot away from what the key/value state store turned into: a far more use-case governed platform with runtime shims holding together what providers failed to provide.
I would dramatically prefer that if there's an AI use-case, it be proposed as a separate block - more of a composite component that's able to express the individual functionality but which only applies to those providers that can do both operations. This is a central tenant of my handily rejected planes proposal, but is exactly along the lines of what I was thinking.
Some people will just need a vector store. Some people will just need a search store. For those that need both, build something that specifically speaks to that use-case but atop far more generic basic components.
|
|
||
| This is a proposal surrounding the implementation of two new specialised state stores without bringing into | ||
| scope the deep integrations planned on implementation of these building blocks. Whilst the two building | ||
| blocks share almost the same envelope, the method of requesting the records can vary significantly and should |
There was a problem hiding this comment.
One thing I'd love to see explicitly spelled out is the story for someone who wants to use both search & vector against the same provider. From ^ it sounds like the answer is to register 2 components pointing to the same server.
When that happens, is the underlying data shared - so an item indexed for search and its vector can be tied together by a common ID or are they independent? I think that matters with AI use cases where you might want to store something once and both search it by keyword && find it by similarity.
And is there a scenario where a single query should do both at once - keyword & similarity combined by the provider itself?
| ### Search | ||
|
|
||
| ```protobuf | ||
| rpc IndexDocumentsAlpha1(IndexDocumentsRequestAlpha1) returns (IndexDocumentsResponseAlpha1) {} |
There was a problem hiding this comment.
Should we also consider additional rpcs for CreateIndex, CreateCollection, DescribeIndex, ListIndexes, DeleteIndexes, etc?
4 participants
Purely a building block implementation of the search and vector stores