Post

alright here it is, my hit-piece on CARs in atproto, with a big ecosystem survey

I think this is legit to call out! STAR-lite is cool and solves a couple other paper cuts. but I think churning on low-level formats in the protocol spec itself probably isn't worth it? I do think the ecosystem can+should migrate away from older flexible IPLD libs to stricter DASL/atproto libs

pretty similar to the relay comparison that @iame.li did a couple months ago. with a bit of attention/coordination I don't think it will be that big of a lift to nail these down. getting the details nailed down in the IETF text first feels like the right sequencing.

yeah, if the reached-for libraries trend toward DASL (with actually-checked CID behaviour, which isn't technically required from the library itself in DASL) then hopefully the safety story gets better. i don't think the footgun will ever go away but maybe a kind of herd immunity works out

for public atproto this was really part 1 -- i think the streaming story is still bad, and that we need changes to DASL. (it doesn't actually say you're allowed to require a specific block order! it also vaguely hints that duplicate blocks could become disallowed! i worry about "happens to work")

end of the day CARs are mostly fine, probably. documenting the warts for me is as much about documenting what we accept as it is about pushing for it to change. (and about answering "why" for STAR-lite and justifying its slight network-fracturing effect)

so is this also how you export and import spaces? because that circles back around to "are permissioned repos nominally stored as a merkle tree or nah" question

export for sync ya and no, afaik just a set of key/values and a 2kb lthash digest over it. you could keep it as an mst for key-value representation if convenient (but i’d argue that for public repos, storing the mst at all is just a convenience too hehe)

I think the IETF version will be a small v4 bump of the repo format, and folks could look for that in the CAR commit block (eg, by sniffing the first 10 KByte of the CAR). is that enough, or do you think we'd still need an HTTP header or something?

i think a v4 needs to require stream ordering (so that 10k sniff works, maybe this is what you’re saying), or else it needs an explicit flag

for a flag, http response header avoids doing gross car hacks (magic root, magic first block, additional field on commit thats not part of the signed rep, …), but a little unfortunate to tie it to transport imo maybe @dasl.ing would be open to somehow marking a car as “ordered”

yeah that is what I was thinking, v4 would always require it