Post

There'll be a split PROPL this year; a morning set of position papers (including climate modeling), and then a working group where we'll put concrete action items together for what the PL/systems community can work on together on!

We need to get out into the wider world and consider the incentive structures for who will actually use these systems. The benefits of this new structure cannot just go towards those regions that don't need it help.

Icechunk is a cloud-native Rust-based chunked, compressed storage format for ndarray. It's got a hierarchical data model which cherrypicks from the best ideas in netCDF and so on and crucially promotes rigor via version control, isolated transactions and snapshots, says @deepakcherian.bsky.social

0 reposts1 like

Since it's a big storage system, stability is super important for a cloud-native weather format. Public providers are moving to it, for example for weather forecasts and cloud data. It supports 'virtual' datasets built from piles of other data so you can compose vast datasets via 'symlinking'

Enzyme works at the LLVM layer so it works on any language and also takes advantage of LLVMs optimisation passes! Avoids derivates of values that don't make sense, and then allocates shadow memory for every differentiable instruction and calculates the derivate from the partial. Performance is good!