Post

hmm, that's where I'm glad to self host my own server. A bit unfortunate the folks on matrix.org homeserver had 2 days of downtime for now.

If instead of just having to type a server yourself in the setup process you had 5-10 alternatives from long-lasting servers (servers.joinmatrix.org) the matrix.org server would not be as crowded and: 1. Users won't suffer from all outages 2. matrix.org would not be such a target for attacks ...

Except that with mastodon your home server doesn’t bind you if you later decide to move on to another instance or even self-host. Migration is painless and you don’t lose any contacts either. That alone takes a lot of stress out of picking where to put your account initially.

I had a server running. But I had to go offline again; the server's resources couldn't handle the load caused by Synapse.

Too bad it's not possible to move to self-hosting without creating an entirely new account.

They did a great job but the best is, we see, all selfhostet instances work further .. What a great success story! Hope you all the best for your restore! Best!

You can use another matrix server: github.com/JokerGermany...

Yeah I was just responding that it's possible for many "rm" cases. Aware of ZFS license restrictions (shame!). I've had good results with BTRFS but I'm sure ZFS is more stable. I would assume they're using a db partition. That said, I run VMs on BTRFS with nocow and performance is good for my needs.

btrfs is pretty cool, and I use it a lot, but both btrfs - and other filesystems or storage with differential snapshotting - have a performance hit that really hurts databases. On updates, small chunks of data need to be moved, causing fragmentation plus metadata overhead from tracking each piece.

a literal “glitch in the matrix” ;>

nope; different physical boxes. things went on the primary while rebuilding the secondary after its RAID failed.

yay, i witnessed a part of matrix history

Decentralized messenger moment. One matrix.org fell - 90% of all rooms are unavailable. To access the rest, you need to register a new account 😆

Any updates?

Why is the database that large? That seems like a lot for just text.

I hope your DB is within those limits? www.postgresql.org/docs/current...

Other servers are unaffected of course, but despite many years of development Element still lacks multi account support.

Status update: we’re 47TB through restoring the 55TB db snapshot of the matrix.org DB, but then have to rebuild the DB and replay the subsequent 17h of DB traffic, which will take several hours. Thank you for your patience, and apologies once again for the outage.