Egor Tarasenko
Egor Tarasenko
@dataish.bsky.social
Data & AI Engineer building scalable data platforms and AI systems with real-world impact. https://www.linkedin.com/in/egorseno/
7 followers2 following134 posts
dbt SQL isn't SQL, it's Jinja2 -- graph tools see {{ ref() }} as a black box, not a dependency. dbt already computed the DAG in manifest.json. dbt-graphify just reads it instead. newsletter.ponder.co/p/dbt-meets-... #dbt #graphify
DuckDB now runs inside the actual dbt Fusion engine, not just the old adapter -- full local lineage, zero warehouse cost. Real catch: dev/prod become different SQL dialects, not just different manifests. Test before you trust. Full breakdown: t.me/broken_pipel... #dbt #DataEngineering
Custom dbt schemas + --defer aren't two separate tricks for local dev and CI -- same manifest, same contract. Watch out though: dbt Core and Cloud don't always generate identical manifests. Full breakdown: t.me/broken_pipel... #dbt #DataEngineering
"Core vs Cloud" is dead. Fusion is Apache 2.0 -- free, fast, prod-ready. Paid: orchestration, Mesh, Semantic Layer, RBAC. t.me/broken_pipel...
Heard about PGQ, Postgres 19's new graph queries? Tempted to rebuild dbt-kg on it instead of FalkorDB, paired with pgvector. newsletter.ponder.co/p/the-dbt-ai...
Text answers from your data are nice. Charts are nicer. We taught our Cube + RAG stack to render BI charts straight in the chat β ask in plain English, get a governed, non-hallucinated graph in ~11 seconds. Video + writeup: telegram.me/broken_pipel... #ConversationalBI #DataEngineering
"Core or Cloud" is the wrong question -- it's not one decision, it's five: development, execution, CI, orchestration, metadata. Nothing forces you to answer them all the same way. newsletter.ponder.co/p/dbt-core-v... #DataEngineering #dbt
Airflow 3.3 fixed the retry problem every data engineer has been burned by... You know this one. A task submits a Spark job. The job runs fine, but the worker dies -- OOM, node rotation, whatever. Airflow retries the task, and it submits a NEW one.... t.me/broken_pipel...
data products vs semantic layers β not the same thing saw a good post from Tim Gasper on this and it stuck with me. his framing: data products are about the promise (who owns it, what quality to expect, who to call when it breaks), semantic layers are about the meaning.
so the real question isn't "should we build a semantic layer", it's "do we keep letting it live in 40 dashboards or do we make it define-once". we went the define-once route: metrics in dbt yaml -> auto-synced to Cube.js and BI -> even queryable back inside the dbt DAG.
Wrote about the whole setup a while back, still one of my favorite things we built: newsletter.ponder.co/p/the-end-to... and yeah β for AI you need both. without data products the LLM cooks with unreliable ingredients, without semantics it doesn't know what the ingredients even are.
data products vs semantic layers β not the same thing saw a good post from Tim Gasper on this and it stuck with me. his framing: data products are about the promise (who owns it, what quality to expect, who to call when it breaks), semantic layers are about the meaning.
the part that hits hardest tho β every org already HAS a semantic layer. it's just accidental. it lives in tableau dashboards, random SQL queries, spreadsheets and tribal knowledge. Self-service BI gave everyone the power to define "customer" slightly differently, and boy did they use it.
Right now it's just me. But the goal is bigger -- more authors, more shared experience, a community of people who love this work as much as I do. To share, progress together, and provide data to this new beautiful world. t.co/H2QobenBKg Stay tuned. There will be more.
dbt + Cube + Superset give you a semantic layer that doesn't rot. The question we kept getting: why stick Cube between dbt and the BI tool at all? Metrics get defined once and reused everywhere -- instead of five dashboards each inventing their own "revenue." newsletter.ponder.co/p/from-seman...
Not indecision -- decomposition. The question isn't which tool your team can handle, it's which layers you need to own. Full version: t.me/broken_pipel... #DataEngineering #dbt
Fable 5 is back after 18 days. The "jailbreak" was reading code and finding bugs -- GPT-5.5 and even Haiku could already do it. The real news: Anthropic now owes the gov't pre-release access + jailbreak disclosure, permanently. Users pay via usage cuts. Enforcement first, rules after. #AI #Anthropic
Ok this one's an experiment. I write long, I know. So here's the same idea in two minutes, TikTok-cut, in case your attention span is also cooked from doomscrolling π« t.me/broken_pipel... Original post: newsletter.ponder.co/p/airflow-ch...
OpenSpec sat in my feed for 4 months. I kept scrolling -- I've watched too many "spec-driven" frameworks become folders nobody opens. Then I had 3 people and an agent in one dbt repo and nobody could say what our models were even supposed to do. So I finally tried it.
Simpler than the hype. Two folders in git: specs/ is the truth, changes/ is work in flight like a feature branch. What won me over: spec review is separate from code review. A spec diff shows what behavior changed -- readable in seconds, not a 600-line PR at 2am.
The catch: it only knows about changes that go through it. The second a teammate edits a model and merges to main, the spec doesn't move -- your source of truth quietly starts lying. So: how do you handle manual vs agent changes in one repo? openspec.dev
π 100 subscribers on our Substack! Three depths, take your pick: π¦ Here: quick hits π¨ Telegram -- longer insights: t.me/broken_pipel... π Substack -- full deep dives: newsletter.ponder.co Thanks to everyone reading π
We built an AI chat for dbt that understands your project as a knowledge graph. It was powerful. But it understood relationships, not meaning. So we fixed that π newsletter.ponder.co/p/the-dbt-ai...
We built an AI chat for dbt that understands your project as a knowledge graph. It was powerful. But it understood relationships, not meaning. So we fixed that π newsletter.ponder.co/p/the-dbt-ai...
We built an AI chat for dbt that understands your project as a knowledge graph. It was powerful. But it understood relationships, not meaning. So we fixed that π newsletter.ponder.co/p/the-dbt-ai...
Government directives don't come with SLAs. Stripe had Fable 5 migrate a 50M-line Ruby codebase in a day. That gets built into roadmaps. And then it's gone on a Friday evening with no timeline. Full breakdown: t.me/broken_pipel...
semantic layers keep solving the definition problem. nobody solves the query interface problem. www.linkedin.com/posts/burakk...
dbt projects become dependency webs nobody can navigate. We loaded the full dbt graph into a graph DB and put an AI chat on top. Ask "what's the most reused source?" and actually get the right answer. newsletter.ponder.co/p/graph-powe...
Open-source pipeline: dbt -> Cube -> Superset, fully automated. No duplicated metric definitions, no manual sync. Add RAG on top and your users query data in plain English -- grounded in actual business logic. newsletter.ponder.co/p/dbt-cubejs...