Post

Hate to say it, but I might need to stop my indexer for docs.surf. It’s. Fun little project, but the costs are slowly creeping upwards as standard.site continues to grow. Tap listening to every single record just eats up way too much egress.

docs.surfDocs.surfAggregator for publications and documents on the open web

bryan newbold

@bnewbold.net

9 Apr

with tap bandwidth do you mean ingress or egress? the full firehose does add up to a lot of ingress every month, but that is no-cost with many hosting providers. in theory you could split out just the tap part to one of those, and keep the rest of your setup where it is

Steve Simkins

@stevedylan.dev

9 Apr

That’s kinda where I ended up. I’m currently running tap on railway and sending webhooks to Cloudflare to handle data processing, but since it’s getting so many records it’s pushing egress costs up.

Steve Simkins

@stevedylan.dev

For a while I moved the tap instance to my home server which did work, but eventually the ingress was so high my home internet was getting throttled (kids and wife were not happy lol).

9/4/26, 3:12 am0 reposts2 likes

Steve Simkins

@stevedylan.dev

9 Apr

I’m likely going to look at quickslice since it can use jetstream instead, so might be going back to a local setup if it’s not too intense. I think even then I might have to filter out bridgy records because there’s just so many.