Mike Driscoll

In data analytics, we're facing a paradox. AI agents can theoretically analyze anything, but without the right foundations, they're as likely to hallucinate a metric as to calculate it correctly. They can write SQL in seconds, but will it answer the right business question?

DuckLake is a simpler, SQL-friendlier alternative to Iceberg. “There are no Avro or JSON files. There is no additional catalog server or additional API to integrate with. It’s all just SQL.“ That said, choose your catalog database — a single-point of failure — *very carefully*.

Quack... Quack... and code! @mehdio.com and @medriscoll.com from @rilldata.com are diving into how GenAI is reshaping BI-as-code — from idea to implementation. This one’s for data folks who want to see beyond the hype. Register : lu.ma/w4ncmttn

Apache Pinot is one of the world’s fastest and most scalable real-time analytical databases, relied on by LinkedIn, Uber, and Stripe. It was awesome diving into the secrets behind its unique architecture with creator and @startreedata.bsky.social founder Kishore Gopalakrishna.

Just published: Ever had to «Scale beyond Postgres»? You may have started with a simple ETL pipeline and crunched critical business logic into useful dashboards, but speed and scale didn't grow with data at some point, and it's the concurrent user. ✨ Below are some highlights from the article.

The father of relational databases understood semantic data models.

Blogged: Exploring UK Environment Agency data with @duckdb.org and @rilldata.com rmoff.net/2025/02/28/e... #dataBS

Why Pivot Tables Never Die A brief history of software’s longest-lived and most-loved data tool, from Lotus 1-2-3 and Excel to QlikView and PowerBI, by @ssp.sh www.rilldata.com/blog/why-piv...

DuckCon #6 is now live from Amsterdam (link below!) In about 20 minutes I'll be sharing some work we've been doing at @rilldata.com on metrics-layer-powered dashboards. And speculating in my last slide, because why not, about what you would name an AI agent that runs on DuckDB... 🦆

“Pivot tables are just a shorter, fatter GROUP BY”. ❤️ this by @gregat.es

DuckCon #6 will start in 168 hours (January 31, 15:00)! If you plan to attend in-person, please register at duckdb.org/2025/01/31/d... The stream will be available without registration.

#databs welcomes the creator of Pandas, founder of Datapad & Voltron, and all around nice guy Wes McKinney @wesmckinney.com to BlueSky.

I'm collaborating with @ssp.sh on a brief history of pivot tables. We'll be tracing their lineage and evolution across Visicalc, Lotus, Excel, PowerPivot, Qlik, and PowerBI. Any sites, videos, products (dead or alive) that you would recommend we should mention or dig into?

Real-time user experiences make applications magical. Google, WhatsApp, and ChatGPT would all fail if you added just 10 seconds to every search, message, or prompt interaction. We are surrounded by fast user experiences and yet most of us tolerate business dashboards that take minutes to load.

It took 5 minutes to create this dashboard with @rilldata.com's AI auto-generate features. Impressive @medriscoll.com!

👋 @davistreybig.bsky.social Welcome! All, Davis is a VC at innovationendeavors.com and someone that shares my vision around object storage. We also co-invested in responsive.dev together! 😁 Here's one of his posts. You should follow him.

I’ve often said working in data engineering is like working in the post office. Queues get backed up, deliveries are lost, yet the stream never stops. It’s a thankless, mostly invisible job until something goes wrong, & then it’s complaints. I hoped the analogies stopped there, but sadly maybe not.

I had the pleasure of talking with @medriscoll.com on his podcast. I love this long-form discussion, especially on a snowy day like this :) Talked about: > My journey (DE ↠ Author) > Bluesky🦋, DuckDB, S3, data modeling & declarative stacks > What do you call data people? 🎙️ youtu.be/xW_HGb46xMk

Snowflake’s pricing is a riddle wrapped in a mystery inside an enigma.

Nice review of the latest CSV parsing & querying capabilities of @duckdb.org. Parquet is obviously better than CSV because of its stronger types, efficient compression, and support for predicate pushdown. But given CSV is *everywhere*, these improvements are welcome. duckdb.org/2024/12/05/c...

@sfchronicle.com maybe you could find it in your heart to remove the paywall on the TSUNAMI EVACUATION MAP? www.sfchronicle.com/bayarea/arti...

In 2003, Jeff Bezos banned Powerpoint from Amazon meetings, insisting managers craft memos, not fancy bullet points. Today, too many dashboards are Powerpoint decks but for data: overly styled visuals better conveyed by a table of metrics. (Ironically, QuickSight dashboards are among the worst).

The video for my "What Goes Around Comes Around... And Around" talk at CWI is now available: www.youtube.com/watch?v=8Woy... 📊Slides: www.cs.cmu.edu/~pavlo/slide... 📄Paper: db.cs.cmu.edu/papers/2024/...

As an engineer, if I've built something once, the chance to do it again is a gift. That's my theory for 🦋 (and the AT protocol) is succeeding technically. In an existing company, rewrites rarely happen - long-term benefit cannot outweigh short-term cost.

reposting for the 1M people who joined in the last 24 hrs @duckdb.org ecosystem people go.bsky.app/8jPCfvb

ICYMI I recently interviewed the creator of @clickhouse.com, Alexey Milovidov. We dove deep into what ClickHouse is so fast, why AI companies love it (hint: its had a vector data type since 2012), rewriting Zookeeper (yikes), and how math informs database design. www.rilldata.com/blog/rill-cl...