Post

Fred Hebert
@ferd.ca

When writing about Property-Based testing, I found they were much more solid and comprehensive than usual tests, due to their more general (properties vs examples). They weren’t used because they felt more complex and time-intensive to write, despite being shorter and having to change them less.

Nathan
@natfu.be

I've read a couple papers on property based testing and the examples are usually very DS&A oriented because it's where testing invariants and defining properties is easy. For business rules it's much more difficult to pin down exact behavior. Requirements might not be precise enough.

Fred Hebert
@ferd.ca

Yeah the invariant-based stuff has somewhat low ceilings in what you can do, but stateful properties have an incredible amount of expressiveness for larger scale integration tests around whole workflows that would in theory be very powerful as a backstop for LLM-based development.

0 reposts2 likes

Gordon Guthrie
@foundationsofthedigitalstate.com

my favourite property based testing is Identity diff and patch for example A = diff(B, C) therefore C = patch(A, B) generate a million sets of B and C and you are away obvs there arent always Identities lying around but when there are - did 8m tests at bet365 based on recording and playback

alias (in chains)
@antiali.as

If I’m understanding quickly, testing by generating arbitrary reified operations and applying them to both a target system and a dual model of the system, then asserting some observations, is that close?

Fred Hebert
@ferd.ca

Yeah. The model is an in-memory test approximation of what should be seen, and the same operations are applied to the real system under test. You then check that the SUT behaves the way the test model states it should. So you can iteratively develop a sort of model/spec along with the real code.