Rebuilding Embeddings

[converseKarl]

If I want to clear out everything and rebuild all the embeddings what is the best approach?

For example a website may change over time and you may want to clear out the website embeddings or the entire thing and rebuild it weekly, monthly etc

[adhityan]

Hello, you can do that with the following methods available within RAGApplication

1. deleteEmbeddingsFromLoader(uniqueLoaderId, true) - this can be used to remove the embeddings from a single loader
2. deleteAllEmbeddings(true) - this will remove all the embeddings from all loaders

The parameter true is constant and is used to ensure you really mean to delete the embeddings. Not passing it or passing false would print a console warning without deleting the embeddings.

Hope this helps.

[converseKarl]

is there anyway to add a webload (url dynamically) incrementally whilst keeping existing loadings, and update the build? something along these lines?

    await ragApplication
        .setTemperature(0)
        .addLoader(new WebLoader({ url: url })) // Use the extracted URL
        .setVectorDb(new LanceDb({ path: './db' }))
        .build();

[adhityan]

Yes. Once you get an instance of ragapplcation, you can call the addLoader method on it at any time to add a new loader. In fact, I would recommend you add all loaders dynamically after build.