Post

fig (aka:[phil])
@bad-example.com

bad news and good news intuition was right: nearly 90% of MST CIDs in the firehose are rebroadcasts of recently-emitted CIDs. bad: an LRU cache needs to be kind of big to get a high hit ratio (128 entries = 4.5KB/repo) good: keeping CIDs from the prev commit only yields 60–70% hit ratio

fig (aka:[phil])@bad-example.com

what if part of sync state was a small per-repo LRU cache of block CIDs? since most updates (at least with tid rkeys) stay on a few hot edges of the mst, maybe you could avoid sending most CIDs with most commits, just referencing a cache index instead

0 reposts25 likes

fig (aka:[phil])
@bad-example.com

CIDs-from-previous-commit doesn't necessarily mean smaller per-repo state, but it is a *much* simpler per-repo state, which i think can be synchronized and evicted in some kind of neat ways.

fig (aka:[phil])
@bad-example.com

i think there's a mostly-unexplored design space here between massive client fan-out and point-to-point links. eg. i want to run my own relay, but not from home, and ideally not with the full weight of multiple full subscribeRepos coming into my home.