Almog

A deep dive to how metrics are stored and queried. Inverted indexes, roaring bitmaps, and ... gorillas? www.bitsxpages.com/p/how-metric...

The team worked insane hours to get this release ready and we've finally shipped it! We built a first of its kind, prometheus-compatible database that is object native. ~22x cheaper than AWS AMP and stupid easy to operate. www.opendata.dev/blog/introdu...

I know "measure twice, cut once" applies to performance optimizations, but where's the fun in that? I am quite proud of my lopsided planter boxes and custom memory allocators.

Databases have strange economic incentives that end up making them "worse" over time. The problem is an asymmetry between who benefits from innovations at first (database companies) and who benefits in the long run (hyperscalers that sell hardware). Blogged: www.bitsxpages.com/p/the-broken...

I regularly get asked "how do you create your diagrams for your blogs"? Recently I've been using my own tool, and I'm open sourcing it! Yuzudraw is a visual editor for ASCII that has a token-efficient DSL that lets agents generate/modify your diagrams. Give it a ⭐! github.com/agavra/yuzud...

“Gradually, then suddenly.” That’s how adoption works when you’re building something new. Opendata is still "gradually" but 100 stars with $0 spent on marketing is a good start. Back to building! github.com/opendata-oss...

I implemented prefix compression for SlateDB & noticed benchmarks looked "worse". Fell down a rabbit hole. Turns out I was thinking about compression backwards... Wrote up my learning: www.bitsxpages.com/p/the-mathem...

Can you beat 180KB? I created a challenge to reduce a dataset as much as possible 🏆 my approach uses delta encoding & prefix compression + zstd(22) compression to reduce 25MB -> 180KB github.com/agavra/bit-g...

I'll die on this hill: Sorted String Tables (SSTs) are the single most important data structure for modern DBs. They lean in to the limitations of SSDs and Object Storage, making them (and similar layouts) the best choice for many databases. Blogged in detail: www.bitsxpages.com/p/sorted-str...

The inner join between sets of people who build databases, write, and draw? Low cardinality. I'm in that set, so I'm starting a blog! Here's my first post: www.bitsxpages.com/p/frameworks...

Sometimes the best solution is "do nothing", but it's always more fun to play with tools.

Calling database nerds in SF! I'm covering SlateDB at the systems meetup next Wednesday (10/29). If you're around, I'd love to meet you in person (that way you'll have proof I'm not just an AI bot). 👉 luma.com/e7feg2i6

Beans on toast is underrated.

I recently implemented Gorilla encoding (www.vldb.org/pvldb/vol8/p...) for a SlateDB PR. Pretty cool stuff - easy to understand but really powerful. Here it is, explained by a gorilla.

It's true. Every company eventually becomes a database company.

Despite using so many new technologies, I somehow never learn my lesson: read the docs sooner and read the docs thoroughly.

Reading the tokio-rs async documentation makes me feel like...

One day I'll open a coffee shop dedicated to the not-insignificant intersection between database nerds and coffee snobs. Until then, enjoy this comic.

Maybe... just maybe, adding more features and complexity into stream processors is NOT what we need?

The new electric Caltrain cars have WiFi. 🙏 SF bay area has finally entered the 21st century (on this dimension of public transit only).

Today marks SlateDB’s one year anniversary! It’s been a lot of fun. Thanks to @rohanpd.bsky.social @flaneur2024.bsky.social @almog.ai @vigneshc.bsky.social @paulbutler.org Jason Gustafson, David Moravek, and many others for joining the project. 😀

"Kafka Configs: A Rube Goldberg Machine?" - an actual quote from a customer after discussing cleanup.policy and Kafka Streams changelogs.

SlateDB now has clones 🤯 Users can clone an existing DB's data to a new location. It's nearly instantaneous since it references the data from the old bucket rather than copying. Writes to the clone update the new location. Compaction lazily merges old data into the new directory./ht @responsive.dev

Should you use Kafka? If so, when? And what are the tradeoffs presented by the dizzying variety of Kafka-adjacent technologies? I hope that my latest blog post provides unique and useful answers to these questions. Let me know what you all think! 👉 www.responsive.dev/blog/why-whe...

As part of my post-Elden Ring hunt for a game, I tried Balatro last night. I'm starting to think I may have made a mistake.

How did Responsive simplify its architecture and achieve monstrously high availability with unlimited scale potential? Co-founder @almog.ai will share how replacing RocksDB with #ScyllaDB in #Kafka Streams led to big wins for their team at Monster Scale Summit. www.scylladb.com/monster-scal...

My hobbies all have the ability to "reclaim" waste: 🪴 for ceramics, wasted clay from throwing goes right back in to your next pot 🪵 for wood working, reusing 2x4s is second nature 🥘 for cooking, leftover food is tossed into salads 💻 what's for software? bin-packing containers? 😆

Kafka Streams 101: Windows & Time! 🕰️ What's the difference between event, stream and wall clock time? 🪟 What are the four different types of windows? ⁉️ What are the important error messages and metrics, and what do they mean? 👉 Read the full lesson here: www.responsive.dev/blog/windows...

🙏 A @notion.com feature on my wishlist: @jetbrains.com IDEs have this features that lets you select the open file in the nav hierarchy. I've always wanted to be able to do this in Notion so I can more easily explore without navigating away from a page.

Kicking the tires on animated explanations of windowing strategies in Kafka Streams. Would love feedback on if this animation for session windows resonates: they keep growing until there is an 'inactivity gap' (no events for the key). This here shows a session window with inactivity gap of 10.

There are some learnings that only come from having seen a wildly successful technology go from a nascent project to ubiquitous. These insights can change the trajectory for new tech. Jason's summary about error handling is a great read for practical engineers: github.com/slatedb/slat...

Nice short read. Got me wondering if we should support SI in SlateDB. So far I’ve been pretty hardline on just SSI.

I just discovered @localfirst.fm and can’t help but compare the parallels between that movement and the resurgence of popularity of embedded data systems & BYOC (or even on prem) tech. “Serverless” has its uses, but the limitations are increasingly clearer.

I wish @xkcd.com published an autobiography. Somehow their work constantly hits bullseye and I’d love to get just a glimpse of the story behind the person that generates creativity in the same league as history’s greatest artists.

[1/5] Hey fellow Kafka Streams otters! You may have already seen my previous blog posts such as the "Debugging Rebalances" and "EOS guide" from last year (and if you haven't, go check them out! Links in thread) But now it's 2025, and time for me to think about what to write next...

Single-writer systems are 🤩 - they transform traditional transactions into a much simpler problem: fencing zombies. This allows read-modify-write without coordination, instead relying on commits to discard data if its fenced (rare). SlateDB enforces single-writer. Let the use cases "sync" in 😉

I've learned more about ACID in the last two days reviewing the discussion on github.com/slatedb/slat... than I have in the last decade in industry. A big takeaway is that using object store for durability opens a new can of worms that existing systems can ignore (high write latency).

Interviewer: Can you explain this gap in your resume? Kafka Streams: I was taking some R&R… rebalancing and restoration.

Kafka Streams has officially come to Bluesky! Follow this account for updates on everything Kafka Streams and keep up to date with new features, bug fixes, and upcoming KIPs. Long live the otter!

Kafka Streams: 25 improvements in time for 2025! Big wins in performance (KIP-925: Rack Aware Assignment), customization (KIP-924: Custom Task Assignment, KIP-954: Custom Storage) and monitoring (KIP-1091: thread state metrics) - just to name a few. See them all: www.responsive.dev/blog/kafka-s...