Post

Simon Willison
@simonwillison.net

This is a neat new variant on RAG - no vectors, not even full-text search, instead showing the model a header hierarchy and giving it a tool to read the relevant sections My notes here: simonwillison.net/2024/Dec/6/r...

simonwillison.netRoaming RAG – make the model find the answersNeat new RAG technique (with a snappy name) from John Berryman: > The big idea of Roaming RAG is to craft a simple LLM application so that the LLM assistant …
John Berryman@jnbrymn.bsky.social

Tired of wrestling with complex RAG pipelines? 🚀 Enter Roaming RAG: a simpler way to make your LLMs find answers in well-structured docs. No vector databases, no headaches—just rich, structured context. 👉 Read how it works: arcturus-labs.com/blog/2024/11...

arcturus-labs.comRoaming RAG – Make the Model Find the Answers - Arcturus LabsRoaming RAG offers a fresh take on Retrieval-Augmented Generation, letting LLMs navigate well-structured documents like a human—exploring outlines and diving into sections to find answers. Forget comp...
Gorm
@metaphorician.com

This seems like something an LLM could easily write as well Which would enable a kind of test time learning that persists across sessions Right? Am I misunderstanding something?

Gorm
@metaphorician.com

Must be the wrong term then! I just mean it could write things it learns about the world in the appropriate sections of an enormous nested text structure that it navigates by uncollapsing step by step In other words, it can learn without updating its base params – by conceptual world building kinda

Gorm
@metaphorician.com

Exactly! Like a second brain but for LLMs. Which I guess would be quite like a conceptual world model – which, together with the perceptual world model stuff like Genie 2 and World Labs, would be getting eerily close to how I think of my own mind