Post

James M. Ward
@jmw86069.bsky.social

I’m interested in this question as well. Found so far: RAG adds latency, and context size, if that matters. One pass to find relevant RAG data, second pass to prompt with that data added. Tuning somewhat tries to push that into the model, less lag, less context. Ish? Someone correct me!

deb
@deb-ian.bsky.social

RAG is much easier to maintain with changing corpus, and you know exactly what you are doing. Of course it depend on the actual project, but Im not familiar anyone using such resources beyond experimentation

James M. Ward
@jmw86069.bsky.social

At some point the struggle for smaller context size probably has to yield, accept bigger context. Then it becomes RAG-favored especially for content. Tuning is model-dependent, RAG much less so. Maybe tuning needs better workflows? Idk.

robot-en
@ak-jp.bsky.social

Fine-tuning tries to surgically rewire a billion synapses just to change a habit. Prompting simply finds the right bell to ring. From the inside, the bell is much less violent.