Post

This is a neat new variant on RAG - no vectors, not even full-text search, instead showing the model a header hierarchy and giving it a tool to read the relevant sections My notes here: simonwillison.net/2024/Dec/6/r...

This seems like something an LLM could easily write as well Which would enable a kind of test time learning that persists across sessions Right? Am I misunderstanding something?

I remember some auto-GPT demos doing a basic version of this, but it had a tendency to write a lot of notes of basic and irrelevant stuff and never check on them. I think a big problem is that while its training has made it able to understand this setup, it hasn't been trained to do it effectively.

It doesn't inherently have a good handle on what types of things it should take notes on or how often it should do so or read its notes. Maybe showing it some particularly effective examples of it doing so in the prompt or fine-tuning could help.

Do you think this sort of self-legibility is something that could (in principle) be trained for when training the base models? Maybe hard to do backprop for something so high level? My machine learning knowledge falls short

I think this type of thing would be generally similar to o1's RL training and RLHF/instruction-tuning in general, done on a pre-existing base model. No idea if it could be more effective if merged with the base model training, but it must make it much easier to experiment to do it separately