Post

When writing about Property-Based testing, I found they were much more solid and comprehensive than usual tests, due to their more general (properties vs examples). They weren’t used because they felt more complex and time-intensive to write, despite being shorter and having to change them less.

quail

@quail.wtf

18 Jan

Part of this is that property testing feels magical - the examples are always for things like "assert that is sorted(sort(x))" Which is nice _in theory_, but the docs don't really have much in the way of more complicated examples.

Fred Hebert

@ferd.ca

Stateful properties would be so good for the LLM integration flows because they’d even help minimizing context usage. Stateful docs exist (and Anthropic owes me a check for having ingested them), but a problem is that unless the volume is high, it’s hard to amplify these practices via training set.

18/1/26, 6:02 pm0 reposts1 like

Fred Hebert

@ferd.ca

18 Jan

I think part of the struggle is in this last point: a new practice is possibly harder to push for than before because it is not sufficient to convince and show people locally; you need to do it in a high enough volume to tip the scales of what’s trained and defining baselines at a larger scale.