Jordan Isaacs

Jordan Isaacs

@snowytrees.dev

database internals engineer; building lakebase @ databricks; making transactions go vroom; author of too many nix flakes

77 followers45 following44 posts

Jordan Isaacs reposted

You're probably right to pick a modern extension to support vector similarity search in Postgres. But did you know Postgres already has one built in? I took a look at the cube extension in Postgres, pgvector, and model2vec for some impressively fast embeddings generation.

Finished the overhaul of kernel-development-flake ✨ inspired by virtme-ng which didn’t support NixOS. kdf is also a rust-based /init using virtiofs. You can now do fast live dev of kernel modules/in-tree, no more waiting for nix rebuilds github.com/jordanisaacs...

github.comGitHub - jordanisaacs/kernel-development-flake: Nix flake for linux kernel development (formerly kernel-module-flake)Nix flake for linux kernel development (formerly kernel-module-flake) - jordanisaacs/kernel-development-flake

TIL rust's spawn implementation uses a unix socket for communication between child & parent (github.com/rust-lang/ru...). Only happens when it can't use posix spawn (e.g. using pre_exec callback). Hit this when running a kernel without unix sockets in qemu and got function not implemented 🫠

Jordan Isaacs reposted

Today's Future Data Systems Seminar Speaker: Joyo Victor will present @singlestore.com's "Bottle Service" meta-data system that supports database branching, change-data-capture, and Apache Iceberg. Zoom talk open to public at 4:30pm ET. YouTube video available after: db.cs.cmu.edu/events/futur...

db.cs.cmu.edu[Future Data] Storage Metadata for Modern Cloud Databases - Carnegie Mellon Database GroupIn modern database architecture, separating compute from storage unlocks powerful capabilities. Our... Read More +

Been working on using github.com/backtrace-la... which does cross modifying code for negligible overhead flags. But when I tried to use it I got dreaded linker errors with the '__start' and '__stop' symbols for the custom sections that were supposed to be generated at link time missing...

github.comGitHub - backtrace-labs/dynamic_flag: A C library for runtime-flippable feature flags on Linux/x86-64, with negligible overhead in the common caseA C library for runtime-flippable feature flags on Linux/x86-64, with negligible overhead in the common case - backtrace-labs/dynamic_flag
View full thread

So after a day of work I had my 3 line fix :) Add 'SHF_GNU_RETAIN' flag to the inline asm for the main flag list. And a dummy variable in the macro that uses the view sections to force a GC reference: `__attribute__((retain,used,section("view-section"))) static const char dummy[0];`

Been working on using github.com/backtrace-la... which does cross modifying code for negligible overhead flags. But when I tried to use it I got dreaded linker errors with the '__start' and '__stop' symbols for the custom sections that were supposed to be generated at link time missing...

github.comGitHub - backtrace-labs/dynamic_flag: A C library for runtime-flippable feature flags on Linux/x86-64, with negligible overhead in the common caseA C library for runtime-flippable feature flags on Linux/x86-64, with negligible overhead in the common case - backtrace-labs/dynamic_flag
View full thread

Luckily I had two build flavors with one passing and the other failing. I manually went through every linker flag difference to guess which one was causing issues. And '—gc-sections' looked suspicious. The linker has some crazy complexity to how it decides to GC sections.

There has actually been a ton of consideration to handling these sort of “meta” sections that have no direct access and thus do not participate in general GC references. But upon further looking the dynamic flag didn’t use any of the special flags. So I found my bug, the linker was GCing it

Been working on using github.com/backtrace-la... which does cross modifying code for negligible overhead flags. But when I tried to use it I got dreaded linker errors with the '__start' and '__stop' symbols for the custom sections that were supposed to be generated at link time missing...

github.comGitHub - backtrace-labs/dynamic_flag: A C library for runtime-flippable feature flags on Linux/x86-64, with negligible overhead in the common caseA C library for runtime-flippable feature flags on Linux/x86-64, with negligible overhead in the common case - backtrace-labs/dynamic_flag

But the sections existed in the object files leaving me flummoxed. Why wasn’t the linker using the sections? So I went down one of the most interesting systems rabbit holes I’ve had recently.

So dynamic flag automatically generates sections using inline assembly. One section has the list of all flags. Then there are arbitrarily many other sections that are a “view” into that list of flags for more efficient lookups. There is no access of any sections beyond using '__start' and '__stop'

Experimental off-cpu profiling landed for otel ebpf profiler 👀. It also was a blocker for using its stack unwinding as a library. Hopefully that means user space stacktraces and profiling will come soon(ish) to inspektor gadget for interactive perf analysis on kubernetes. github.com/open-telemet...

github.comOff CPU profiling by florianl · Pull Request #196 · open-telemetry/opentelemetry-ebpf-profilerThis is the code that backs #144. It can be reused to add features like requested in #33 and therefore can be an alternative to #192. The idea that enables off CPU profiling is, that perf event and...

You are up to date.