Post

Has anybody in my network set up local notes (Markdown files) with an Ollama RAG/model trained on such notes? So far, below is the best shot I found. I want to play around over Christmas break and wonder about the best approaches. I collect some ideas here: www.ssp.sh/brain/second.... #obsidian

Yes, I have tried it with Flowise as well as building it from scratch using Python, Ollama and LangChain. I loaded the documents to the vector store separately and then used them in my inferencing. If you know the structure of your markdown, you can use it to better guide the chunking process.

I haven't done any additional fine tuning, and used existing fine tuned models.

What is total dataset size in bytes? If complex reasoning across the whole set of notes is required for your use case — it could be! — RAG will fall over on you.

Don't you think that context size and the use of tools may play a role here before we start talking about fine tuning? Some of the reasoning/aggregation can be done outside the context and then the results can be injected into the context.

Depends on the use case. If the query is “what is my most controversial opinion across all my notes?” then rag can easily fall over unless you anticipated it ahead of time in the indexing pipeline. That’s admittedly an extreme example, but the spectrum between that & simple fact retrieval is blurry

Also, if data is converted to triples, the use of Graph RAG techniques can help us run Graph queries to query and filter what is needed and only inject that into the context. Thoughts?

if this command is correct, I have 32 MB of text (without images or other fieles): ❯ find . -name "*.md" -type f -exec du -ch {} + | grep total$ 32M total

Can you also provide the count? Because it does seem like 32MB is a lot for 7k files.

Storing vector embeddings is like storing records in a database and the number of your notes will not matter. However, it does take time to vectorize it and load them into the database. Having a fast GPU or lot of CPU will help! Fine tuning is not required in these cases. RAG must be sufficient.

Alright, that is great insight. Thanks so much for sharing. I also have all connections and backlinks from my Obsidian Vault in a singular JSON file, so I guess this would also help any algorithm or RAG learn/improve relations and connections. I will try to find a way to use that, too.

I suggest, as a Hello World, take a subset of the markdown documents and add them to the vector store and see if you are able to use them in your query context.