Post

Opening with @neurocy.bsky.social explaining the 'working group' model we're doing for the 3rd PROPL

There are clear roles for computer scientists to have a role in managing planetary health, and we can win! #propl26

There'll be a split PROPL this year; a morning set of position papers (including climate modeling), and then a working group where we'll put concrete action items together for what the PL/systems community can work on together on!

While there are many digital commons out there for scientific data, the future digital commons must also *compute* using FAIR principles, says @neurocy.bsky.social doi.org/10.1145/3759...

By FAIR, we mean good support for findability, accessibiilty, interoperability, reproducibility, while supporting liveness at scale.

We need to get out into the wider world and consider the incentive structures for who will actually use these systems. The benefits of this new structure cannot just go towards those regions that don't need it help.

Next up, @deepakcherian.bsky.social presents Icechunk, a storage system that scales with the whole planet!

Deepak's a great example of someone who wouldn't ordinarily attend @sigplan-pldi.bsky.social; he's a 'professional scientist' who has worked on terabyte scale oceanic models, and now maintains important scientific infrastructure like xarray

Oceanography has an inspiring history of open source from the 90s zenodo.org/records/1061... but more sophisticated ideas and services never built a network effect and so we've ended up with 'standard compliant self-describing metadata-rich piles of files'

Icechunk is a cloud-native Rust-based chunked, compressed storage format for ndarray. It's got a hierarchical data model which cherrypicks from the best ideas in netCDF and so on and crucially promotes rigor via version control, isolated transactions and snapshots, says @deepakcherian.bsky.social

Since it's a big storage system, stability is super important for a cloud-native weather format. Public providers are moving to it, for example for weather forecasts and cloud data. It supports 'virtual' datasets built from piles of other data so you can compose vast datasets via 'symlinking'

@deepakcherian.bsky.social notes that Icechunk requires a level of homogeneity which doesn't work well for some data. Need cross-language FFIs (something @neurocy.bsky.social also observed). Earth sciences is missing database ideas like incremental view/streaming compute and derived fields

Next William S. Moses talks about an enormously ambitious compiler-first planetary compute engine! How can we automatically differentiate in a performance portable Earth System Modeling engine? agupubs.onlinelibrary.wiley.com/doi/10.1029/... - Core problem is differentiation is compute intensive

Enzyme works at the LLVM layer so it works on any language and also takes advantage of LLVMs optimisation passes! Avoids derivates of values that don't make sense, and then allocates shadow memory for every differentiable instruction and calculates the derivate from the partial. Performance is good!

Enzyme is useful in real climate problems like the shallow water equations. It saves both execution time and storage costs by smart checkpointing in the compiler pipeline and can run for large tasks. Scientists don't write TPU code typically, but code can be compiled using a parallel.for in Enzyme

Enzyme is full of optimisations like communications fusion and low precision arithmetic towards order of magnitude speedup when running on large deployments. Compilers are a promising route to avoid earth scientists to become domain scientists in high performance computing!

@dorchard.bsky.social advertises the @iccs.bsky.social 'hacktoberfest' to collaborate with people on climate science projects; sign up later this year!

@dorchard.bsky.social then makes the case that general purpose programming languages are impeding our scientific communication! That's because complexity often obscures meaning in our source code. For example, fluid simulations often need to be rewritten for a different hardware architecture

"Mind the gap" applies to science says @dorchard.bsky.social: code has an abstract model that's lost on the way to getting results. Why don't we communicate via the code? Programming languages are *too* general purpose to help here. Names have no intrinsic meaning in PLs but are helpful to humans.

@f.luid.org language moves beyond notebooks to explain itself via literate execution. CamFORT tries to bring units of measure to code. But many of these are partial solutions; what would a complete language solution look like? Fortl is @dorchard.bsky.social's new language! github.com/plas4sci/fortl