Post

Post deletes being complete happening soon! Thanks @jay.bsky.team! NOTE regardless of network, scraping happens and can't really be stopped. For years a bot reposted my every tweet. Please be careful. BlueSky is a playground - not a different universe - even if we rejoice in momentary reprieve.

yes, people should treat data here like it’s the public web. we had a long discussion today about whether/how to discourage scraping/archiving — @pfrazee.com?

yeah I'm gonna go Big Technical on this one

every user in atproto has a signed data repository, which is quite similar to a git repository. changes get written in commits and each repository maintains a history (commit log) we do that because the network is largely pull-based, and maintaining the history makes catchup sync efficient

designing the data repository is a bit of an art, because it's a merkle tree and you're balancing the size of proofs for a given record or subset of the data against the flexibility of the data it contains. @dholms.xyz is your guy on how all that works

as it turned out, the way image blobs get stored in the repository make it easy to just drop them and leave dangling references. that's why we're able to ship that quickly records, however, are harder to expunge without losing the ability to verify the proofs

daniel tells me it might be possible to change the data structure so that it's possible, but he had that kind of quiver in his voice when he said it that tells me I shouldnt expect it any time soon that means if we want to expunge the actual records, we have to do a "rebase"

a "rebase" basically means rewriting the history to remove any record of the past writes. generally speaking you want to do this every so often as a form of compaction, because otherwise every repository is going to grow unbounded. so, that's something the team designed for from the start

This smells like the kind of thing that will lead to something like “The Great Skeet Massacre” or “Rebaseageddon” around 2027

Rebasing sounds a little bit like garbage collection

I suppose it’s more complicated than marking deleted entries as “dirty” and preventing aggregators from collecting those nodes or something at the protocol level

I'd be curious how bsky team thinks about long term viability of deletes. I generally assume AT is incentivized to be widely insta-scraped, given the low latency requirements of the medium making it possible and the incentives of the "AI dark forest" hunting for alpha, public shaming targets, etc.

I think having a delete flag makes sense, so "good faith" aggregators and clients can respect it to reduce blast radii. But the idea of actually implementing a real physical purge operation as a privacy feature doesn't seem like it is gonna move the needle on anything real in the long run.