Post

what if part of sync state was a small per-repo LRU cache of block CIDs? since most updates (at least with tid rkeys) stay on a few hot edges of the mst, maybe you could avoid sending most CIDs with most commits, just referencing a cache index instead

we debated a similar optimization at some point, I think why came up with it based on filesystem implementation tricks: have a small array of records be referenced directly (by path and CID) from the commit object. they would still go out on firehose, but wouldn't churn the MST structure

the firehose event would be just the new (signed) commit object, and the (new?) records. no MST nodes. the motivation was this was a high rate of small interaction records such as "likes".

I personally didn't like this because it adds implementation complexity in a bunch of places (PDS, consumers, anybody requesting a proof chain). you need to be very careful doing existence lookups, handling invalid corner-cases, etc. I just think correctness+reliably would suffer in practice

(I'd be more amenable if the repo data structure was truely an internal implementation detail and there was only 1-2 actual implementations. but we want many implementations!)

my counter-proposal is to have the PDS (or just client SDKs) support "linger" as an optional flag on record creation. the PDS would cache the record, and only include on firehose once a non-linger commit happens (or, eg, a timeout of a few minutes). this works b/c most ops are clustered in time