This page was generated from notebooks/remote_datasets.ipynb. Interactive online version: Binder badge.

Remote datasets

Pyfesom2 comes with remote datasets that are loaded on demand.

[2]:

from pyfesom2 import datasets

1. Tutorial datasets

A smallest and quick to load dataset

[3]:

datasets.tutorial_dataset.load()

[3]:

<xarray.Dataset>
Dimensions:  (elem: 5839, nelem: 5727, nod2: 3140, nz: 48, nz1: 47, three: 3, time: 2)
Coordinates:
    lat      (nod2) float64 dask.array<chunksize=(3140,), meta=np.ndarray>
    lon      (nod2) float64 dask.array<chunksize=(3140,), meta=np.ndarray>
  * time     (time) datetime64[ns] 1948-12-30T23:45:00 1949-12-31T23:45:00
  * nz1      (nz1) float64 -2.5 -7.5 -15.0 ... -5.525e+03 -5.825e+03 -6.125e+03
  * nz       (nz) float64 0.0 -5.0 -10.0 -20.0 ... -5.65e+03 -6e+03 -6.25e+03
    faces    (nelem, three) uint32 dask.array<chunksize=(5727, 3), meta=np.ndarray>
Dimensions without coordinates: elem, nelem, nod2, three
Data variables:
    a_ice    (time, nod2) float32 dask.array<chunksize=(1, 3140), meta=np.ndarray>
    m_ice    (time, nod2) float32 dask.array<chunksize=(2, 3140), meta=np.ndarray>
    temp     (time, nod2, nz1) float32 dask.array<chunksize=(1, 3140, 47), meta=np.ndarray>
    u        (time, elem, nz1) float32 dask.array<chunksize=(1, 5839, 47), meta=np.ndarray>
    v        (time, elem, nz1) float32 dask.array<chunksize=(1, 5839, 47), meta=np.ndarray>
    w        (time, nod2, nz) float32 dask.array<chunksize=(1, 3140, 48), meta=np.ndarray>
Attributes:
    Dataset URL:  https://swiftbrowser.dkrz.de/public/dkrz_035d8f6ff058403bb4...

A CMIP6-like and a more comprehensive dataset.

[12]:

datasets.core.load()

[12]:

<xarray.Dataset>
Dimensions:  (nod2: 126858, nz1: 47, time: 144)
Coordinates:
    lat      (nod2) float64 dask.array<chunksize=(126858,), meta=np.ndarray>
    lon      (nod2) float64 dask.array<chunksize=(126858,), meta=np.ndarray>
  * nz1      (nz1) float64 -2.5 -7.5 -15.0 ... -5.525e+03 -5.825e+03 -6.125e+03
  * time     (time) datetime64[ns] 1948-01-31T23:15:00 ... 1959-12-31T23:15:00
Dimensions without coordinates: nod2
Data variables:
    temp     (time, nod2, nz1) float32 dask.array<chunksize=(1, 126858, 47), meta=np.ndarray>
    salt     (time, nod2, nz1) float32 dask.array<chunksize=(1, 126858, 47), meta=np.ndarray>
    a_ice    (time, nod2) float32 dask.array<chunksize=(1, 126858), meta=np.ndarray>
    m_ice    (time, nod2) float32 dask.array<chunksize=(1, 126858), meta=np.ndarray>
    ssh      (time, nod2) float32 dask.array<chunksize=(1, 126858), meta=np.ndarray>
    sst      (time, nod2) float32 dask.array<chunksize=(1, 126858), meta=np.ndarray>
Attributes:
    Dataset URL:  https://swiftbrowser.dkrz.de/public/dkrz_02942825-0cab-44f3...

2. Frontier simulation examples

Pyfesom2 comes with examples of frontier simulations

A sample Arctic 1km resolution dataset with ~11 million surface nodes.

[9]:

datasets.arctic_1km.load()

[9]:

<xarray.Dataset>
Dimensions:  (nod2: 11538465, nz1: 69, time: 60)
Coordinates:
    lat      (nod2) float64 dask.array<chunksize=(100000,), meta=np.ndarray>
    lon      (nod2) float64 dask.array<chunksize=(100000,), meta=np.ndarray>
  * nz1      (nz1) float64 0.0 -5.0 -10.0 -15.0 ... -5.4e+03 -5.65e+03 -6e+03
  * time     (time) datetime64[ns] 2006-02-28T23:58:00 ... 2011-01-31T23:59:00
Dimensions without coordinates: nod2
Data variables:
    a_ice    (time, nod2) float32 dask.array<chunksize=(1, 100000), meta=np.ndarray>
Attributes:
    Dataset URL:  https://swiftbrowser.dkrz.de/public/dkrz_02942825-0cab-44f3...

A sample Rossby 4.2 simulation with ~23 million surface nodes and 79 levels.

[8]:

datasets.rossby42.load()

[8]:

<xarray.Dataset>
Dimensions:  (nelem: 46260679, nod2: 23243988, nz: 80, nz1: 79, three: 3, time: 1)
Coordinates:
    faces    (nelem, three) uint32 dask.array<chunksize=(361412, 1), meta=np.ndarray>
    lat      (nod2) float32 dask.array<chunksize=(363188,), meta=np.ndarray>
    lon      (nod2) float32 dask.array<chunksize=(363188,), meta=np.ndarray>
  * nz       (nz) float32 -0.0 -5.0 -10.0 -20.0 ... -5.8e+03 -6.05e+03 -6.55e+03
  * nz1      (nz1) float64 -2.5 -7.5 -15.0 ... -5.7e+03 -5.925e+03 -6.3e+03
  * time     (time) datetime64[ns] 1948-01-01
Dimensions without coordinates: nelem, nod2, three
Data variables:
    temp     (time, nod2, nz1) float32 dask.array<chunksize=(1, 100000, 40), meta=np.ndarray>
Attributes:
    Dataset URL:  https://swiftbrowser.dkrz.de/public/dkrz_035d8f6ff058403bb4...

As these remote datasets are currently statically served there are other versions of this rossby4.2 that are optimized level: datasets.rossby42_level and spatial: datasets.rossby42_spatial based access.

3. AWI-CM CMIP6 grids

These datasets are just coordinates corresponding to LR, MR, HR versions of AWI-CM model used in CMIP6. Currently these contain face information that contain missing information in ESGF and necessary for spatial plotting. Soon using diagnostics from pyfesom2 may be supported on these datasets.

Merging this grid data with CMIP6, AWI-CM can be done using xarray.merge([grid_data, cmip6_data]).

[14]:

datasets.cmip6_hr.load()

[14]:

<xarray.Dataset>
Dimensions:  (depth: 47, ncells: 1306775, nelem: 2581792, nz: 48, three: 3)
Coordinates:
  * depth    (depth) float64 -2.5 -7.5 -15.0 ... -5.825e+03 -6.125e+03
    faces    (nelem, three) uint32 dask.array<chunksize=(161362, 1), meta=np.ndarray>
    lat      (ncells) float64 dask.array<chunksize=(81674,), meta=np.ndarray>
    lon      (ncells) float64 dask.array<chunksize=(81674,), meta=np.ndarray>
  * nz       (nz) float64 0.0 -5.0 -10.0 -20.0 ... -5.65e+03 -6e+03 -6.25e+03
Dimensions without coordinates: ncells, nelem, three
Data variables:
    *empty*
Attributes:
    Conventions:  CF-1.7
    Dataset URL:  https://swiftbrowser.dkrz.de/public/dkrz_035d8f6ff058403bb4...

Use datasets.cmip6_lr.load() and datasets.cmip6_mr.load() for LR and MR respectively

4. FESOM-like dataset

Most useful for developers to test, benchmark new methods on FESOM-like grids, that may not be available locally. If dask is available then a data variable is returned along else only coordinate dataset is returned. Check method doc for more use cases.

[17]:

datasets.fesom_like(100000)

[17]:

<xarray.Dataset>
Dimensions:    (nelem: 199964, nod2: 100000, three: 3)
Coordinates:
    lon        (nod2) float64 -55.58 -81.23 -106.1 92.0 ... -130.4 155.1 86.75
    lat        (nod2) float64 -45.61 -65.2 -15.11 -62.37 ... 48.31 38.89 55.55
    faces      (nelem, three) int32 14415 15036 50348 ... 52107 18289 71945
Dimensions without coordinates: nelem, nod2, three
Data variables:
    fesom_var  (nod2) float64 dask.array<chunksize=(100000,), meta=np.ndarray>
[18]:

datasets.fesom_like(100000, times=10, levels=10)

[18]:

<xarray.Dataset>
Dimensions:    (nelem: 199972, nod2: 100000, nz1: 10, three: 3, time: 10)
Coordinates:
  * time       (time) datetime64[ns] 2010-01-31 2010-02-28 ... 2010-10-31
  * nz1        (nz1) float64 0.0 -666.7 -1.333e+03 ... -5.333e+03 -6e+03
    lon        (nod2) float64 60.3 21.67 -60.87 49.23 ... 43.86 -52.57 46.77
    lat        (nod2) float64 -86.05 -89.97 39.7 43.34 ... 30.54 1.721 63.64
    faces      (nelem, three) int32 74935 93671 69952 24049 ... 55770 75732 1512
Dimensions without coordinates: nelem, nod2, three
Data variables:
    fesom_var  (nz1, time, nod2) float64 dask.array<chunksize=(10, 10, 100000), meta=np.ndarray>