Post

what does memory mean for ai? for large language models, it still mostly feels like language and text. but the kind of “experience” that turns a junior dev into a senior one—built over time, through mistakes—often can’t be captured in just words.

we train ai for long hours hoping it can retain and reuse what it learned. just like how we hope senior devs’ experience can be passed down. but what kind of memory is that, really? and how can it be preserved and reused over time?

It feels something like this. LLMs have the knowledge part, and sometimes the intuition. However they often don’t have the wisdom bit that prevents them going wildly off course, down dead ends and ultimately failing.

Or maybe it doesn’t even simulate the insight bit; but instead just kind of ambles aimlessly along the knowledge graph approaching something close to a useful solution.