Diptanu Choudhury

Diptanu Choudhury

@diptanu.bsky.social

CEO @tensorlake.bsky.social Past - AI Infrastructure at Facebook, LinkedIN, Hashicorp, Netflix

2K followers112 following71 posts

Diptanu Choudhury
@diptanu.bsky.social

I need to post here more often! We have been working on sandboxes for agents @tensorlake.ai and have built the fastest sandbox infrastructure that is out there. Fastest file system, consistent sub second startup times and stateful resume and suspend of sandboxes for durable agents!

Diptanu Choudhury
@diptanu.bsky.social

If you are building agents, would love to have a conversation. While many companies in this space focuses on developer experience, we are also focused on performance and consistency beyond the DX!

Diptanu Choudhury
@diptanu.bsky.social

Some more color on this. What does it mean to help models solve interesting problems with MCP? MCP is the easiest way to give models access to the gazzilion APIs enterprises have already built. yes, it’s a security nightmare but nevertheless it helps teams with the Day 1 problems of 🚢 MVPs

Diptanu Choudhury@diptanu.bsky.social

I was a skeptic until I heard banks and hedge funds using MCP to enable models solve interesting business problems. And yes OpenAPI + tool calls would have just worked fine but developers like frameworks and abstractions for building the first MVP. MCP being more prescriptive thank OpenAPI helps.

Diptanu Choudhury
@diptanu.bsky.social

I was a skeptic until I heard banks and hedge funds using MCP to enable models solve interesting business problems. And yes OpenAPI + tool calls would have just worked fine but developers like frameworks and abstractions for building the first MVP. MCP being more prescriptive thank OpenAPI helps.

Chris@chris.blue

New post! MCP seems like a lot of hype without much substance. Seems like some developers are catching on.

materializedview.ioThis MCP Server Could Have Been a JSON FileThere's a lot of buzz around MCP. I'm not convinced it needs to exist.
Diptanu Choudhury
@diptanu.bsky.social

One of our customers told me today the biggest lift for structured extraction with @tensorlake is that their engineering team can now tweak the schema they want to extract from documents every week as they evolve their insurance platform. These little things make AI find roots in enterprises.

Diptanu Choudhury
@diptanu.bsky.social

Here's an instance where Gemini Flash 2 misinterpreted a table, sourced from a residential property's mold report. It mistakenly attributed mold data from outside the property to a bedroom, as it failed to parse some rows from the third column

Diptanu Choudhury
@diptanu.bsky.social

This would be a problem if you used the data in an agentic application for a real estate business. This is why Table Structure Understanding models are still the most reliable way to parse tables in business critical applications.

Diptanu Choudhury
@diptanu.bsky.social

The emergence of browser agents makes me feel we are lacking a google for APIs. Instead developers are making LLMs search for things, and turning websites into APIs by making LLMs click on the UI like humans. This approach is going to take a long time to get right, and wasteful for a lot of reasons

Diptanu Choudhury
@diptanu.bsky.social

Structured Extraction is essential for AI engineering teams, we are now making it faster and more reliable than ever, whether you're turning PDFs, invoices, or reports into structured data. Here is a sneak peak into our Structured Extraction engine.

Diptanu Choudhury
@diptanu.bsky.social

Test prompts instantly with our intuitive JSON schema editor and extract complex tables, figures, and charts with ease. Once your schema is set, run our API at scale—tens of thousands of times per day—ingesting new documents and writing structured data directly into your database or warehouse.

Diptanu Choudhury
@diptanu.bsky.social

Python Folks - which data/workflow engine has the best developer experience for packaging code? We have looked into - Modal, Beam, Airflow, Flyte, AWS Lambda, Prefect, Dagster and Spark. Haven’t seen any approach which is fast, reliable and intuitive.

Diptanu Choudhury
@diptanu.bsky.social

We have been using O1 or Sonnet to solve a problem to understand the upper bounds of what models are capable of, and falling back to our internal models or open source models for economy and security. Been working pretty well, is this a common workflow?

Diptanu Choudhury
@diptanu.bsky.social

Turned on Apple Intelligence this morning. We are a long way from having a personal assistant on the iPhone! It wish it summarized all unread from Slack, Gmail, WhatsApp and messages and came up with a list of things I needed to respond :)

Diptanu Choudhury
@diptanu.bsky.social

Qwen2VL 72B is just better than every other closed and open source vision model for document understanding. Like every other vision model, it's still incapable of retaining every single ground truth on dense documents.

Diptanu Choudhury
@diptanu.bsky.social

Throwing the kitchen sink at a small problem. Whenever I work on an Applied AI problem I work with unconstrained compute to see if we can solve a business problem if money was not a constraint. If there is enough value in solving the problem, the economy of scale can kick in later.

Diptanu Choudhury
@diptanu.bsky.social

Building a solid compute engine is time and capital intensive - Probably a big reason why we see execution engines use SQL as a front end. They can re-use some parts of the planner and the DSL. But then they make the trade off of pushing SQL in domains where it doesn’t make sense.

Diptanu Choudhury
@diptanu.bsky.social

If you are processing unstructured data(document, audio, raw text, etc) - you really want to write Python or Typescript with the ability to load data from blob stores, and have a GPU attached to the runtime to be able to leverage AI models. SQL is a big impedance mismatch for these use cases.

Diptanu Choudhury
@diptanu.bsky.social

I love that DMs are open to chat with people on BSKY by default! Had some great conversations with folks today! Please DM if you are working on anything related to unstructured data, LLMs and Document Understanding!

Diptanu Choudhury
@diptanu.bsky.social

Looking forward to this! We just started using SlateDB in @tensorlake.bsky.social’s compute engine, looking forward to not needing dynamodb for CAS. One less moving part! @chris.blue when is the new release coming? :)

Chris@chris.blue

And it's done. Thank you so much @benesch.bsky.social for doing this. Now SlateDB will be able to have CAS for all major object stores. 🔥 It just works.

github.comobject_store: Conditional put and rename_if_not_exist on S3 · Issue #6285 · apache/arrow-rsIs your feature request related to a problem or challenge? Please describe what you are trying to do. This is now supported on S3: https://aws.amazon.com/about-aws/whats-new/2024/08/amazon-s3-condi...
Diptanu Choudhury
@diptanu.bsky.social

The OCI ecosystem for building and distributing containers have improved SOOO much in the last 4-5 years! It's mind blowing. Cold Startups for large containers are almost not a problem anymore if you have enough network bandwidth and some CPU to spare.

You are up to date.