Opening with @neurocy.bsky.social explaining the 'working group' model we're doing for the 3rd PROPL
Post
While there are many digital commons out there for scientific data, the future digital commons must also *compute* using FAIR principles, says @neurocy.bsky.social doi.org/10.1145/3759...
Next up, @deepakcherian.bsky.social presents Icechunk, a storage system that scales with the whole planet!
0 reposts0 likes
Deepak's a great example of someone who wouldn't ordinarily attend @sigplan-pldi.bsky.social; he's a 'professional scientist' who has worked on terabyte scale oceanic models, and now maintains important scientific infrastructure like xarray
Oceanography has an inspiring history of open source from the 90s zenodo.org/records/1061... but more sophisticated ideas and services never built a network effect and so we've ended up with 'standard compliant self-describing metadata-rich piles of files'
Icechunk is a cloud-native Rust-based chunked, compressed storage format for ndarray. It's got a hierarchical data model which cherrypicks from the best ideas in netCDF and so on and crucially promotes rigor via version control, isolated transactions and snapshots, says @deepakcherian.bsky.social