Tigris Data

The core parts of relational databases are transactions, tables, and indices. At Tigris we use a non-relational database (FoundationDB) but still have indices. How did we do it? We made our own!

Hey all, we have more updates on objgit! Looks like it's taking off amidst other companies making git services for some reason. Again, really have to wonder what that's all about.

Either way we want to keep chugging along at this and turn objgit into the git server that we wish we had: infinitely scalable and with all the important data on object storage. As an added bonus we'd also try to make a scraper-resistant web UI.

Hey all, we have more updates on objgit! Looks like it's taking off amidst other companies making git services for some reason. Again, really have to wonder what that's all about.

We want to make something for your private repos to stay that: private. You don't want to have to be woken up because some scraper decided your git repos were fresh tokens ripe for training.

Hey all, we have more updates on objgit! Looks like it's taking off amidst other companies making git services for some reason. Again, really have to wonder what that's all about.

Tigris is basically a database company that turned into an object storage company, so we have a lot of opinions about what you can do with them. We use FoundationDB as the backbone of our business. Wouldn't it be great to insert message queue items in the same transaction as inserting data?

We fixed that by adding Kafka to the stack. Kafka has a...reputation for being a bit grotestue in the infrastructure side of things (the zookeeper can't even pet the animals 😔), but we've found that using it means we take write pressure off of FoundationDB so the servers stay up.

Tigris is basically a database company that turned into an object storage company, so we have a lot of opinions about what you can do with them. We use FoundationDB as the backbone of our business. Wouldn't it be great to insert message queue items in the same transaction as inserting data?

In a slightly dystopian bent, workers own no jobs yet still will be happy as they lease them from the queue by pushing them forward into the future such that other workers don't try to pick them up.

Tigris is basically a database company that turned into an object storage company, so we have a lot of opinions about what you can do with them. We use FoundationDB as the backbone of our business. Wouldn't it be great to insert message queue items in the same transaction as inserting data?

Obviously when we needed a message queue we started by reading a paper by Apple about how they turned FoundationDB into what basically amounts to Kafka. This gives us reliable delivery and scheduling in the same lines of code. It's kinda remarkable how simple it is: jobs identify when they are run.

If our objgit post was so good last time, where's objgit post 2? Today we have an update on the epic saga of Xe reinventing Git storage on top of object storage: making seek-native packfiles. What does this mean? Well, git is complicated. Huddle into this thread!

So Xe invented a new brave format that does store enough information to construct ranged get requests. This ended up making Git object storage on top of Tigris object storage upwards of 4.5-14.6 times faster than the old approach of badly adapting filesystem interfaces to Tigris.

If our objgit post was so good last time, where's objgit post 2? Today we have an update on the epic saga of Xe reinventing Git storage on top of object storage: making seek-native packfiles. What does this mean? Well, git is complicated. Huddle into this thread!

Those objects end up being fairly small, so you'd think "oh, Tigris is good at small objects, I can just put every object into Tigris and then everything will be fine forever with free happy puppies, right?" Turns out, not really because light is only so fast.

If our objgit post was so good last time, where's objgit post 2? Today we have an update on the epic saga of Xe reinventing Git storage on top of object storage: making seek-native packfiles. What does this mean? Well, git is complicated. Huddle into this thread!

A terrible way to think about Git is that it tracks the changes made to an empty folder and through those changes you reconstruct the history of your application's source code.

An agent can run off a VM image that is larger than the disk it runs on. AgentENV by KVCache.AI keeps microVM disk images in object storage and pulls down only the blocks each machine actually needs. That is how image size stops being limited by local disk.

𝗪𝗵𝘆 𝗽𝘂𝘁 𝗮𝗴𝗲𝗻𝘁 𝘀𝗮𝗻𝗱𝗯𝗼𝘅 𝘀𝘁𝗮𝘁𝗲 𝗶𝗻 𝗼𝗯𝗷𝗲𝗰𝘁 𝘀𝘁𝗼𝗿𝗮𝗴𝗲? For one sandbox on one machine, a host mount is fine. It starts to matter 𝘄𝗵𝗲𝗻 𝘆𝗼𝘂 𝗿𝘂𝗻 𝗮 𝗳𝗹𝗲𝗲𝘁, where sandboxes are created and destroyed constantly and land on whichever node has room.

1. With a volume plugin, 𝗰𝗿𝗲𝗱𝗲𝗻𝘁𝗶𝗮𝗹𝘀 𝗻𝗲𝘃𝗲𝗿 𝗲𝗻𝘁𝗲𝗿 𝘁𝗵𝗲 𝘀𝗮𝗻𝗱𝗯𝗼𝘅 and isolation is stronger. The keys stay in a root-owned config on the host. 2. Since mounts are reference-counted per node, 𝗼𝗻𝗲 𝗱𝗮𝘁𝗮𝘀𝗲𝘁 𝘀𝗲𝗿𝘃𝗲𝘀 𝗺𝗮𝗻𝘆 𝘀𝗮𝗻𝗱𝗯𝗼𝘅𝗲𝘀. Model weights mount read-only into every agent on a box from a single copy.

𝗪𝗵𝘆 𝗽𝘂𝘁 𝗮𝗴𝗲𝗻𝘁 𝘀𝗮𝗻𝗱𝗯𝗼𝘅 𝘀𝘁𝗮𝘁𝗲 𝗶𝗻 𝗼𝗯𝗷𝗲𝗰𝘁 𝘀𝘁𝗼𝗿𝗮𝗴𝗲? For one sandbox on one machine, a host mount is fine. It starts to matter 𝘄𝗵𝗲𝗻 𝘆𝗼𝘂 𝗿𝘂𝗻 𝗮 𝗳𝗹𝗲𝗲𝘁, where sandboxes are created and destroyed constantly and land on whichever node has room.

How does Tigris work? What are the moving parts? Today we have a transcript of our CTO Himank's talk about how Tigris works at a high level and how FoundationDB makes it possible.

Delete doesn't have to mean the data is gone. Tigris lets you un-delete your most precious objects and buckets if you(r agent) didn't really mean to actually remove it for others.

If you use RAGFlow by InfiniFlow to answer questions from your documents, everything it knows lives in its storage backend. By default, if you use MiniIO, RAGFlow re-reads your corpus every time you tune it.

@apache.org OpenDAL fully supports Tigris Data. This storage layer with one API across 50+ backends, in 17 languages. Tigris config is just three lines. Get the benefit of multi-region workers, with no cross-region egress charges and no per-region bucket topology to maintain.

1/ The era of manual install scripts for Blessed Frameworks™ is over. New post on `tigris init --agent` and why agent-native onboarding is becoming the baseline.

8/ So tigris init has two doors. The wizard for humans in a shell, where installing to an agent is safe because the agent hasn't started. The --agent recipe in plain text, because text is the only thing that works reliably from inside a session.

1/ The era of manual install scripts for Blessed Frameworks™ is over. New post on `tigris init --agent` and why agent-native onboarding is becoming the baseline.

6/ "Ask before creating access keys" beats "don't create access keys silently." Prohibitions get unreliable when they're buried far upstream of the action. But positive phrasing is a reliability technique. Magic adverbs are not a security mechanism.

1/ The era of manual install scripts for Blessed Frameworks™ is over. New post on `tigris init --agent` and why agent-native onboarding is becoming the baseline.

4/ The agent needs none of that. Worst case it explores the repo and figures out how you build and deploy. Give it the goal, not the mechanism.

1/ The era of manual install scripts for Blessed Frameworks™ is over. New post on `tigris init --agent` and why agent-native onboarding is becoming the baseline.

2/ Your onboarding funnels people into two buckets: users of the framework you picked, and everyone else. Everyone else gets the API reference and good luck. I've been everyone else. I closed the tab.

If you use dlt by dltHub to move data, you can set up the pipeline to save your results into a cloud storage bucket that everything else reads from. Most teams put that bucket on AWS, right next to the pipeline. However, when something reads the data back, AWS charges an egress fee.