Post

I wrote a quick blog post on how to setup and use Amazon S3 Tables with @duckdb.org, based on its new Iceberg capabilities: tobilg.com/query-s3-tab...

tobilg.comQuery S3 Tables with DuckDBDuckDB has gained a new feature in preview, that allows querying of Iceberg data in AWS S3 Tables. Setting up a S3 Table There are multiple steps which need to be performed to set up a S3 Table that can be then queried with tools like DuckDB. As the ...

14/3/25, 6:39 pm2 reposts16 likes

swasheck.bsky.social

@swasheck.bsky.social

14 Mar 2025

this is interesting to me as i have hive-partitioned parquet files in S3 that i use to query directly from dbeaver using duckdb. however, i've not yet come up with a solution for making this information generally available to my team. i thought iceberg would be a good solution

swasheck.bsky.social

@swasheck.bsky.social

14 Mar 2025

however, i struggle with implementing it. so my question here is, how does athena pricing interact with your solution here? is it just s3 storage cost, or would we get hit with data scan charges?

Tobias Müller

@tobilg.com

14 Mar 2025

If you already have Hive partitioned Parquet files that support your main query patterns, then I don’t really see a benefit of moving to S3 Tables/Iceberg at the moment. If you require interoperabilty with e.g. Snowflake, it would eventually make more sense. Regarding pricing, check the AWS docs

swasheck.bsky.social

@swasheck.bsky.social

14 Mar 2025

thank you. that makes sense, i'm just struggling to come up with a sensible solution to allow the rest of my team to access the data (i mean, i suppose i can document using dbeaver+duckdb to query) in a way that provides meaning and insight. thank you again.