Post

Should Kubernetes controllers+CRDs managing resources in the external world (non-k8s APIs) try to fix drifts (such as out of band modifications made in the external API) periodically –or assume they're the only actor, and wait for the next modification to reconfigure? This makes scaling harder.

What makes you think the CR is more correct than reality? IMO once reconciled the operator is done.

CR is the intended config, so I expect it to overwrite the reality –the question is whether it should try to course-correct periodically. There are some systems (like Prodspec/Annealing) that learn from the current state and bring it back to the configuration SoT, but they're few and rare.

My interpretation is that the CR is the intended state during reconciliation, not forever. If something/someone changes the actual state, they'll be surprised to find it asynchronously revert. I don't think the CR can realistically be a SoT most of the time.

Yeah that worries me. My team's operators own their resources -- even we can't manipulate them ourselves. But there is still drift. Some state transitions are irreversible, for example.

Yeah we have a few APIs like that too, they don't play well with the "assume an open world" assumption of KRM. We do our best to validate the state transitions in webhooks. IMO not being able to manipulate them is a feature. :P when we need to manipulate we set a field called `spec.paused:true`.