Post

Figuring out a workflow to make Claude better at creating diagrams and illustrated explanations. By default it is pretty crap - see this example. Labels overlap. Arrows in the wrong place. Text too small. But it's also decent at identifying what's wrong once it takes a screenshot.

Claude's second attempt at fixing some of the problems, with a lot of additional steering from me on improving the visual details. It's still unable to make an arrow go where it needs to, or make two elements connect with a line. But it's a decent starting point - enough for me to cleanup by hand.

My theory here is the major labs have done very little or no RLHF on visual explanations and diagrams? Seems like a blind spot. Image generation alone or text generation alone are very well optimised but the combination seems uniquely hard for current models.

We’ve reached identical conclusions. If you figure this out I’d be very keen to hear how you did it!

Something that helped when I was writing my prompt caching post was constraining it to a smaller grid. I created a little SVG library in React where I could create nested grids for easier positioning and the LLM seemed to appreciate that. It also made it easier to clean up when it got it wrong.

Gemini / Nano Banano became the best at it with the recent large update, any context I'm missing on why to use Claude here?