Post

Steering vectors can work as activation oracles! Inspired by @isolyth.dev 's recent work on instruct vectors, I've ported the same approach to create Activation Oracles. Seems to work surprisingly well given the param deficit, nearly matching Activation Oracle LoRA performance.

One thing, it might be nice to see baseline/completely untrained models for comparison. How good is qwen on its own at this vs vectors?

One one more thing, you might've published vectors I'm not sure, did you check the effects of more/less data? I found that using fewer samples actually helped performance in the assistant task, and as more data was added the vectors got sparser, which makes me think over-optimization/anti-generalize

I mean less data entirely! I found that the vector I got with just 1250 assistant samples was far better than the one trained with 20k, both at 3 epochs

Also what is your batch size?

github.com/LunNova/vect...