Post

@larryosterman.github.io

Larry Osterman@larryosterman.github.io

Oh, and remind me of the time a team I was on used cryptprotectdata to encrypt a secret on a vm managed by vmss. Once a month (reliably), the service failed because the key was corrupted. It took us WEEKS to figure it out.

Larry Osterman

@larryosterman.github.io

OK, so what was happening was that when VMSS handled updates, they decided that rather than run the normal windows update process (which takes the machine offline for some amount of time), instead, they took a clean new vm and copied the user data onto the new vm.

Larry Osterman

@larryosterman.github.io

Which worked incredibly well, but the machine specific crypto keys used by CryptProtectData changed and any data protected with CryptProtectData was now unreadable... And now we know.

2/10/26, 4:21 am0 reposts8 likes

Brian Hoyt

@hoyty.com

This sounds like the magician movie where the trick was making a duplicate rather than teleporting. I mean it is pretty much the same but just a few bits different.

Raj Chaudhuri

@rajch.bsky.social

For some reason, that movie gave me nightmares for days. I know the VMSS story above will, too.