Post

This is a neat new variant on RAG - no vectors, not even full-text search, instead showing the model a header hierarchy and giving it a tool to read the relevant sections My notes here: simonwillison.net/2024/Dec/6/r...

simonwillison.netRoaming RAG – make the model find the answersNeat new RAG technique (with a snappy name) from John Berryman: > The big idea of Roaming RAG is to craft a simple LLM application so that the LLM assistant …

John Berryman@jnbrymn.bsky.social

Tired of wrestling with complex RAG pipelines? 🚀 Enter Roaming RAG: a simpler way to make your LLMs find answers in well-structured docs. No vector databases, no headaches—just rich, structured context. 👉 Read how it works: arcturus-labs.com/blog/2024/11...

arcturus-labs.comRoaming RAG – Make the Model Find the Answers - Arcturus LabsRoaming RAG offers a fresh take on Retrieval-Augmented Generation, letting LLMs navigate well-structured documents like a human—exploring outlines and diving into sections to find answers. Forget comp...

Dan Clark

@danclark.org

Interesting approach. I'd love to see what the results look like One downside to this is, I don't think it would scale with a large amount of documents but it could be great with smaller datasets

6/12/24, 6:17 am0 reposts0 likes

Peter Sergeant

@sgnt.ai

6 Dec 2024

Exactly. Marginal cost of adding a document to pgvector is minuscule, where adding a bunch of tokens for headers to every query will start to add up fast

Maxim

@mzaks.bsky.social

6 Dec 2024

I +1 this answer. Factually we are moving offline preparation to online evaluation and inference. The online eval can lead to better results as it is specific to the context of the prompt at hand, but it needs to burn through lots of cycles. A good offline strategy (if possible) is more efficient.