Merging many vector stores
What are strategies for merging large sets of FAISS vector stores? I’ve used the tool concatenator node for merging smaller sets of stores (see the example image attached below where I’ve connected to stores for use with a query to ChatGTP-4). But as my embeddings become more numerous, sizable and complex, I’m wondering if there is a judicious way to do this in KNIME (i.e., avoiding using the tool concatenator nodes thousands of times!). Or do I need to do this outside of KNIME for now (e.g., using the fromtexts and mergefrom functions from LangChain)? Or am I thinking about all of this too narrowly and missing better strategies entirely?