Post

I’ve been experimenting with feeding around 200 log posts from my site into an LLM to summary, following a certain template structure. The result have been pretty promising with Haiku and Sonnet. With a bit of prompt eng, Gemini also does well.

I ran into problems when trying to make incremental updates using an aider style string replace. Sonnet follows the instructions but seems to lose track of context in the middle. Also, doing this with Claude isn’t cheap. Gemini is about an order of magnitude less expensive than Haiku.

All models I tried seem to struggle with recall for context in the middle. This matters for my task because I am aiming to summarize all the notes for all the months in one inference. I could do month-by-month summarization separately but it undermines the proof of concept I am attempting.

The idea is to take raw notes and express them in several different forms, using an LLM and a template as the mechanism for summarization/transformation. With an initial version generated, hypothetically, I could make updates to the existing document with regenerating the entire document.