Post

Stateful properties would be so good for the LLM integration flows because they’d even help minimizing context usage. Stateful docs exist (and Anthropic owes me a check for having ingested them), but a problem is that unless the volume is high, it’s hard to amplify these practices via training set.

I think part of the struggle is in this last point: a new practice is possibly harder to push for than before because it is not sufficient to convince and show people locally; you need to do it in a high enough volume to tip the scales of what’s trained and defining baselines at a larger scale.

quail

@quail.wtf

At some point the property tests I can come up with just feel like some extra ceremony around example tests, but then I lose nice properties like being able to easily debug!

I’d need to show some examples at work, like everything we have that generated extensive scenarios to walk through a release integration scenario could be given as a recipe for it in stateful property tests and have it shrink the failing steps for you.