Post

jd
@jd.codes

I've been leading the data infrastructure efforts at my job (I used to work as a Data Engineer in big tech) and the stack we've landed on is so enjoyable to work with. @dagster.io - Orchestration @duckdb.org - Database @motherduck.com - Data Warehouse/storage DBT - Data modeling

jd
@jd.codes

We felt that Dagsters focus on materialized assets instead of the DAG itself fit our model better. The baked in metadata management, lineage, and scheduling capabilities were also a big plus. Airflow is incredible software but the overhead for managing these things is too high for our small team.

0 reposts1 like

jd
@jd.codes

Not really. I looked into it but I just had a hard time shifting away from the asset focused model Dagster has to a pipeline focused one like Prefect. I'd like to experiment with it one day though cause it looks cool!