Datasets and repos to be highlighted on website

[NoraLoose]

Here is a list of datasets and repos that we could highlight on the website (including the ones that are already there, but with more meta info). Sometimes there are several repos associated with an item. I suggest to link the one that I did not put into parentheses.

Machine learning tutorials and tools

• Machine Learning tutorial for Lorenz 96:

• Book: https://m2lines.github.io/L96_demo/intro.html

• Repo: https://github.com/m2lines/L96_demo

• Equation discovery:

• Paper: https://onlinelibrary.wiley.com/doi/abs/10.1029/2022MS003258

• Repo: https://github.com/m2lines/EquationDisco

• Geospatial ML prediction workflow

• Repo: https://github.com/anastasiaGor/geoTrainFlow

Software packages

• GCM-Filters

• Paper: https://joss.theoj.org/papers/10.21105/joss.03947

• Documentation: https://gcm-filters.readthedocs.io/en/latest/index.html

• Repo: https://github.com/ocean-eddy-cpt/gcm-filters

• xgcm

• Documentation: https://xgcm.readthedocs.io/en/latest/index.html

• Repo: https://github.com/xgcm/xgcm

Benchmark datasets

• Pyqg parameterization benchmarks (Ross et al., 2023)
• Paper: https://onlinelibrary.wiley.com/doi/abs/10.1029/2022MS003258
• Book: https://m2lines.github.io/MLwithQG/intro.html
• Repos:
  • https://github.com/m2lines/pyqg_parameterization_benchmarks
  • https://github.com/m2lines/MLwithQG
• Datasets: hosted on globus, access as described here, e.g.:

import xarray as xr
import fsspec
# Datasets are hosted on globus as zarr files
def get_dataset(path, base_url="https://g-402b74.00888.8540.data.globus.org"):
    mapper = fsspec.get_mapper(f"{base_url}/{path}.zarr")
    return xr.open_zarr(mapper, consolidated=True)

• Bias corrected ERA5 skin temperature over Arctic sea ice
• Paper: https://doi.org/10.1175/MWR-D-22-0130.1
• Dataset: https://zenodo.org/record/8338265

Datasets accessible in the cloud

• CM2.6

import xarray as xr
import gcsfs
fs = gcsfs.GCSFileSystem()
fs.ls("gs://leap-persistent-ro/groundpepper/GFDL_cm2.6")
ds = xr.open_dataset('gs://leap-persistent-ro/groundpepper/GFDL_cm2.6/GFDL_CM2_6_CONTROL_DAILY_SURF.zarr', engine='zarr')

• MOM6 simulations with ML parameterizations
• simulation with ZB20
• simulation with ePBL_NN
• need status update from @jbusecke

Models and implementation of parameterizations

• Differentiable QG model in pytorch

• Repo: https://github.com/hrkz/torchqg

• Paper: https://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2022MS003124

• Differentiable QG model in JAX

• Repo: https://github.com/karlotness/pyqg-jax/

• Documentation: https://pyqg-jax.readthedocs.io/en/latest/index.html

• Paper: https://www.climatechange.ai/papers/iclr2023/60

• Stacked shallow water model with stochastic subgrid momentum parameterization

• Repo: https://github.com/arthurBarthe/swe_stochastic_param/tree/0.1

• Paper: http://onlinelibrary.wiley.com/doi/abs/10.1029/2021MS002534

• Implementation of parameterizations in pyqg

• Pull request: Add support for online parameterizations / custom forcing functions pyqg/pyqg#266

• Paper: https://onlinelibrary.wiley.com/doi/abs/10.1029/2022MS003258

• Implementation of data-driven parameterizations in MOM6

• Pull request: Implementation of Zanna-Bolton-2020 equation discovery model of mesoscale momentum fluxes. Combined commits. NOAA-GFDL/MOM6#356

• Paper: Perezhogin et al. (2023), in prep.

• Implementation of stochastic parameterization in MOM6

• Code: https://github.com/chzhangudel/Forpy_CNN_GZ21

• Paper: http://arxiv.org/abs/2303.00962

Ocean parameterizations

• Stochastic parameterization of subgrid momentum forcing (Guillaumin and Zanna, 2021)

• Paper: http://onlinelibrary.wiley.com/doi/abs/10.1029/2021MS002534

• Repo:

  • (original: https://github.com/arthurBarthe/subgrid/tree/0.2)
  • (M2LInES fork: https://github.com/m2lines/subgrid)
  • revamped repo (work in progress): https://github.com/m2lines/gz21_ocean_momentum

• Generative data-driven approaches for stochastic subgrid parameterizations (Perezhogin et al., 2023)

• Preprint: https://arxiv.org/abs/2302.07984

• Repo: https://github.com/m2lines/pyqg_generative

• Neural network parameterization for vertical mixing

• Preprint: https://arxiv.org/abs/2306.09045

• Dataset and code: https://doi.org/10.5281/zenodo.7955323

Atmospheric parameterizations

• Neural networks for parameterization of subgrid atmospheric processes (Yuval et al., 2021)

• Paper: https://onlinelibrary.wiley.com/doi/abs/10.1029/2020GL091363

• Repo:

  • (original: https://github.com/yaniyuval/Neural_nework_parameterization)
  • (M2LInES fork: https://github.com/m2lines/Neural_nework_parameterization)
  • revamped repo (work in progress): https://github.com/m2lines/convection-parameterization-in-CAM

• Datasets: https://drive.google.com/drive/folders/1TRPDL6JkcLjgTHJL9Ib_Z4XuPyvNVIyY

• Random forest to learn atmospheric parameterization

• Paper: https://www.nature.com/articles/s41467-020-17142-3

• Repo:

  • (original: https://github.com/yaniyuval/subgrid_parameterization)
  • M2LInES fork: https://github.com/m2lines/subgrid_parameterization

• Neural-network parameterization of subgrid momentum transport in the atmosphere (Yuval and O’Gorman, 2023)

• Paper: https://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2023MS003606

• Repo:

  • (original: https://github.com/yaniyuval/momentum_parameterization/tree/1.1)
  • M2LInES fork: https://github.com/m2lines/momentum_parameterization

• Datasets: https://drive.google.com/drive/folders/1TRPDL6JkcLjgTHJL9Ib_Z4XuPyvNVIyY

Sea ice parameterizations

• Deep learning of systematic sea ice model errors from data assimilation increments (Gregory et al., 2023)
• Preprint: https://arxiv.org/abs/2304.03832
• Repo: https://github.com/m2lines/seaice_DA-ML

[NoraLoose]

@karlotness Is there some code (repo for differentiable model and / or training) related to your recent work that we could highlight on the website?

[NoraLoose]

@aakashsane is there some code related to your recent paper that we could highlight on the website? I'm thinking either a github repo with code for training the NN and / or a PR that implements the NN in MOM6.

[NoraLoose]

@dhruvbalwada have you submitted the ANN module as a PR to MOM6 already? Any other code to highlight from your work?

[NoraLoose]

@chzhangudel have you submitted the GZ21 module as a PR to MOM6? Otherwise / alternatively we can link this repo on the website: https://github.com/chzhangudel/Forpy_CNN_GZ21

[jbusecke]

Are folks using xgcm? Would love to get some more publicity to that if possible.

[NoraLoose]

I couldn't imagine a life without xgcm! 😎 We can add it to the software section.

[karlotness]

For the work I've been doing recently, I haven't opened up the actual experiment code quite yet (but hopefully soon).

The JAX QG model we've been using as part of that project is available though. The code for that is here: https://github.com/karlotness/pyqg-jax/