Example for post-processing output data

This notebook servers as a usage example for data post-processing of the freva plugin climpact. In a first processing step the data has been processed and stored in the cloud by the freva plugin. It is very likely that this data hasn’t been processed sufficiently, e.g unit conversion of the variables is necessary, calculating new variables are. This step, which is highly individual, can be done in this notebook with just a few commands.

Here we assume that the the data has already been produced by the freva plugin and was successfully stored in the cloud.

For working with this data the plugin also comes with a library which creates so called data-containers allowing for easy access of the data in the cloud.

Hence before we start working, we import all necessary libraries.

%matplotlib inline
import xarray as xr
from climpact import DataContainer
import metpy.calc

Calculating the LCL

In this example we are going to calculate the lifted condensation level (LCL) from temperature, pressure and relative humidity. To calculate the LCL we are using the metpy library. First we open the data that has been processed by the plugin and was saved into the cloud. This can be done by the from_zipfile. This method will open the dataset in the cloud, read its content and create a so called data container

data_container = DataContainer.from_zipfile(
    "https://swift.dkrz.de/v1/dkrz_3d3c7abc-1681-4012-b656-3cc1058c52a9/climpact/k204230/ProcessedModelData-nukleus-ger-3km-tdps_ts_ps_pr-20220131_111203T.zip"
)
data_container

Downloading: 100%|██████████████████████████████████████████████████| 72.2k/72.2k [00:00<00:00, 1.66MB/s]

	model name
index
0	ecmwf-eraint-gerics-remo2015_r1i1p1

This convenient html representation of the data-containers content shows the model member(s) belonging to this data container. We can also inspect the variables that are stored within the data container:

data_container.variables

['tdps', 'ts', 'ps', 'pr']

The individual model datasets can be accessed by index or model names according to table above, using the select method. For example:

dset = data_container.select(by_name="ecmwf-eraint-gerics-remo2015_r1i1p1") # or
dset = data_container.select(by_index=0)
dset

<xarray.Dataset>
Dimensions:  (time: 696)
Coordinates:
  * time     (time) datetime64[ns] 2009-06-01T00:30:00 ... 2009-06-29T23:30:00
Data variables:
    height   (time) float64 2.0 2.0 2.0 2.0 2.0 2.0 ... 2.0 2.0 2.0 2.0 2.0 2.0
    tdps     (time) float64 284.4 284.3 284.2 284.1 ... 284.3 283.9 283.8 283.7
    ts       (time) float64 285.9 285.6 285.3 285.1 ... 291.2 290.3 289.6 289.1
    ps       (time) float64 9.87e+04 9.871e+04 9.872e+04 ... 9.861e+04 9.863e+04
    pr       (time) float64 9.504e-05 5.129e-05 2.84e-05 ... 1.101e-11 2.56e-09
Attributes: (12/41)
    CDI:                            Climate Data Interface version 1.9.8 (htt...
    history:                        Mon Mar 29 17:28:32 2021: cdo -O -s -f nc...
    Conventions:                    CF-1.6
    tracking_id:                    ed6f10e1-3780-4fba-a454-110790c8bbc3
    creation_date:                  2021-03-29T17:28:32Z
    CDO:                            Climate Data Operators version 1.9.8 (htt...
    ...                             ...
    ensemble_member:                ecmwf-eraint-gerics-remo2015_r1i1p1
    experiment:                     evaluation
    output_freqeuncy:               1hr
    shape_file:                     /work/ch1187/regiklim-work/k204230/German...
    swift_group:                    ch1187
    path_prefix:                    ProcessedModelData-nukleus-ger-3km-tdps_t...

xarray.Dataset

• Dimensions:
  • time: 696
• Coordinates: (1)
  • time
    (time)
    datetime64[ns]
    2009-06-01T00:30:00 ... 2009-06-...

    standard_name :

    time

    long_name :

    time

    axis :

    T

    array(['2009-06-01T00:30:00.000000000', '2009-06-01T01:30:00.000000000',
           '2009-06-01T02:30:00.000000000', ..., '2009-06-29T21:30:00.000000000',
           '2009-06-29T22:30:00.000000000', '2009-06-29T23:30:00.000000000'],
          dtype='datetime64[ns]')

• Data variables: (5)
  • height
    (time)
    float64
    2.0 2.0 2.0 2.0 ... 2.0 2.0 2.0 2.0

    array([2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
           2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
           2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
           2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
           2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
           2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
           2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
           2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
           2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
           2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
           2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
           2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
           2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
           2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
           2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
           2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
           2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
           2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
           2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
           2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
    ...
           2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
           2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
           2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
           2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
           2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
           2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
           2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
           2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
           2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
           2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
           2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
           2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
           2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
           2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
           2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
           2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
           2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
           2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
           2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
           2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.])

  • tdps
    (time)
    float64
    284.4 284.3 284.2 ... 283.8 283.7

    standard_name :

    dew_point_temperature

    long_name :

    2m Dewpoint Temperature

    units :

    K

    code :

    168

    grid_mapping :

    rotated_pole

    array([284.38294176, 284.26907316, 284.17128302, 284.08838172,
           284.24960639, 284.8534849 , 285.47878534, 285.96015073,
           286.33219306, 286.55964621, 286.65681117, 286.52672796,
           286.23651146, 285.89726048, 285.64625622, 285.45814607,
           285.14839547, 284.7489677 , 284.45997653, 284.29904404,
           284.23293996, 284.20731237, 284.23618321, 284.32835233,
           284.45764321, 284.58534748, 284.68071536, 284.7464653 ,
           284.85002359, 285.08807659, 285.42499157, 285.85631996,
           286.34694272, 286.81693263, 287.18989476, 287.4480638 ,
           287.57537078, 287.57905394, 287.48839949, 287.33855911,
           287.14051688, 286.9030473 , 286.61425862, 286.31244823,
           286.07013278, 285.86056032, 285.67223348, 285.52453889,
           285.34559946, 285.10563565, 284.83660269, 284.62422155,
           284.53839986, 284.64034803, 284.89847763, 285.12704303,
           285.28358556, 285.33983092, 285.34149198, 285.28728371,
           285.16920064, 285.01279633, 284.7948349 , 284.51970118,
           284.22526773, 283.92214947, 283.63420551, 283.37401945,
           283.20089113, 283.0782648 , 282.96603   , 282.85421962,
           282.73516267, 282.61857292, 282.49986998, 282.39425728,
           282.46639602, 282.77750095, 282.99751242, 283.11145506,
    ...
           285.55558524, 285.33134559, 285.00777941, 284.61973092,
           284.30440158, 284.16097273, 284.13252749, 284.14294352,
           284.11831701, 284.06511262, 283.98318886, 283.92299009,
           284.04567221, 284.53195667, 285.20487122, 285.68173891,
           285.94966018, 286.14404108, 286.2891735 , 286.34510321,
           286.25898429, 286.06641634, 285.87848442, 285.69839124,
           285.47328221, 285.23179493, 284.8703178 , 284.49976764,
           284.20859913, 283.9781123 , 283.80781979, 283.71842515,
           283.71187319, 283.69035016, 283.65051348, 283.6011868 ,
           283.73673452, 284.37059919, 285.28646871, 285.99053691,
           286.44058616, 286.5913233 , 286.56356671, 286.46602354,
           286.38089409, 286.25635924, 286.03228479, 285.81881359,
           285.67850275, 285.58289184, 285.20160679, 284.62112541,
           284.22763391, 284.04072283, 283.90006481, 283.74123749,
           283.57131265, 283.38441155, 283.20377366, 283.04812907,
           283.36042243, 284.76916406, 286.68784382, 287.40773468,
           287.52121656, 287.57450904, 287.48536389, 287.34957779,
           287.21857661, 287.07310808, 286.86662338, 286.67670253,
           286.65443568, 286.66905338, 286.00841417, 285.00391955,
           284.28504667, 283.92932819, 283.77511038, 283.72961713])

  • ts
    (time)
    float64
    285.9 285.6 285.3 ... 289.6 289.1

    standard_name :

    surface_temperature

    long_name :

    Surface Temperature

    units :

    K

    code :

    139

    grid_mapping :

    rotated_pole

    array([285.91958805, 285.56647639, 285.25364675, 285.05017438,
           285.56502255, 286.93333535, 288.56591418, 290.22588389,
           291.65479049, 292.7989595 , 293.63089098, 293.83995554,
           293.5527081 , 292.86658889, 292.11712346, 291.28136999,
           290.27737513, 289.15779983, 288.13461858, 287.38386344,
           286.89356449, 286.58655829, 286.37661834, 286.21495115,
           286.07229056, 285.96627031, 285.90703528, 285.88786725,
           286.02573692, 286.411453  , 287.03959387, 287.88528227,
           288.90504156, 289.95646084, 290.78243486, 291.38020951,
           291.68811903, 291.65913912, 291.36319178, 290.86967776,
           290.24737614, 289.45844705, 288.63032382, 288.01796945,
           287.57415371, 287.22178524, 286.96880429, 286.78887733,
           286.61502909, 286.43863508, 286.26423916, 286.11199424,
           286.12730026, 286.56474303, 287.42668842, 288.3896137 ,
           289.30013257, 289.96405285, 290.48075223, 290.85652646,
           290.99627788, 290.91191147, 290.57230609, 290.01329564,
           289.26835049, 288.36220115, 287.36534097, 286.56851953,
           285.97938255, 285.48861995, 285.04673859, 284.64520319,
           284.28379205, 283.97242076, 283.71074802, 283.52372073,
           283.89232031, 284.95336089, 286.41672219, 288.05098124,
    ...
           292.35699229, 291.28888875, 290.04095828, 288.91721535,
           288.14374891, 287.61688041, 287.26748091, 287.01113803,
           286.80937601, 286.65504237, 286.51881451, 286.44542   ,
           286.67436736, 287.32320508, 288.24049843, 289.152264  ,
           290.06611425, 290.89110518, 291.64635254, 292.23534343,
           292.52309785, 292.50638265, 292.35299759, 291.99465225,
           291.35477852, 290.4407007 , 289.31609251, 288.34944591,
           287.66695277, 287.17016547, 286.77615235, 286.48014226,
           286.23368787, 286.03484303, 285.85987065, 285.74305618,
           286.05575688, 287.15774535, 288.75096735, 290.65106052,
           292.45768521, 293.90755453, 295.01811016, 295.68802563,
           296.03892695, 296.08556679, 295.72530271, 295.12756174,
           294.33367826, 293.20314824, 291.5451887 , 289.9934263 ,
           288.92173735, 288.13862686, 287.51855096, 287.00741128,
           286.57356142, 286.20322793, 285.87804455, 285.67396794,
           286.56439164, 289.00704761, 292.06141944, 294.71897967,
           296.74521821, 298.13939152, 298.99986331, 299.45763155,
           299.57530674, 299.43575663, 298.97749479, 298.31828702,
           297.56597664, 296.38487706, 294.38845966, 292.52523929,
           291.22926025, 290.30610627, 289.63819799, 289.14402364])

  • ps
    (time)
    float64
    9.87e+04 9.871e+04 ... 9.863e+04

    standard_name :

    surface_air_pressure

    long_name :

    Surface Air Pressure

    units :

    Pa

    code :

    134

    grid_mapping :

    rotated_pole

    array([98699.50856264, 98709.79332765, 98718.0928215 , 98716.30481209,
           98712.57099082, 98703.71357797, 98681.47191861, 98655.41933115,
           98628.01278791, 98608.42234798, 98586.23680577, 98578.2507184 ,
           98584.67906592, 98563.87472482, 98532.70564703, 98500.84904841,
           98485.91707993, 98488.80247755, 98481.3930393 , 98478.24443773,
           98488.54431468, 98487.25913026, 98461.94921359, 98441.42193373,
           98438.61061717, 98446.392185  , 98445.27829229, 98447.51932551,
           98436.01721139, 98426.92926987, 98413.41447755, 98396.50504485,
           98403.65030879, 98405.56626321, 98406.98105759, 98405.47729836,
           98394.60221622, 98381.38310295, 98369.83924637, 98347.53605896,
           98319.49647209, 98283.89829009, 98261.79824494, 98248.60233724,
           98233.4535333 , 98233.6293826 , 98239.50963915, 98220.60522889,
           98207.95471139, 98192.17729066, 98164.80428257, 98147.52905195,
           98132.39609204, 98114.42126255, 98111.0033309 , 98102.41871056,
           98094.29195791, 98078.43652984, 98068.33523616, 98067.27465447,
           98049.46752787, 98050.98316061, 98062.55449863, 98060.5648151 ,
           98026.61428812, 98011.42677483, 97997.90853038, 97993.17096376,
           97998.2553121 , 97993.80869652, 98011.10367067, 98031.78468506,
           98059.36354809, 98079.5541433 , 98101.29437301, 98108.21820837,
           98121.82207169, 98109.96233521, 98114.08604463, 98121.484078  ,
    ...
           97704.7828961 , 97739.34610339, 97765.34071888, 97814.1632459 ,
           97861.72061057, 97888.73580247, 97919.88912154, 97941.61665857,
           97980.86341973, 98014.99304597, 98038.33158128, 98062.00216968,
           98098.43533588, 98127.44905706, 98159.02550222, 98168.10794951,
           98217.65893214, 98253.15428705, 98274.1376159 , 98301.17070445,
           98336.97755469, 98373.08917178, 98390.45877159, 98393.9585109 ,
           98385.50786031, 98379.68196312, 98378.24667168, 98395.47793839,
           98409.34264274, 98424.52348631, 98453.25828211, 98474.0023653 ,
           98481.26664595, 98497.7658849 , 98523.86033743, 98552.31876738,
           98567.39665906, 98564.4875823 , 98569.76986892, 98575.83925253,
           98596.00402945, 98599.71773659, 98588.45222586, 98590.91667877,
           98577.17732895, 98574.59455351, 98559.09474854, 98546.55616973,
           98541.9840536 , 98525.12189349, 98548.62475379, 98569.74632389,
           98597.22980213, 98615.96513539, 98623.14780382, 98625.42268585,
           98630.22580502, 98635.75449747, 98632.07639021, 98640.63476275,
           98651.00926001, 98669.11571261, 98664.25098286, 98654.34028558,
           98668.28350578, 98673.63726811, 98658.13678859, 98644.60151409,
           98644.03514772, 98655.55588661, 98647.2484642 , 98635.1150821 ,
           98615.03803917, 98601.7232324 , 98598.93270156, 98601.42349789,
           98607.87713654, 98602.43121476, 98608.31991947, 98626.62831879])

  • pr
    (time)
    float64
    9.504e-05 5.129e-05 ... 2.56e-09

    standard_name :

    precipitation_flux

    long_name :

    Precipitation

    units :

    kg m-2 s-1

    code :

    142

    grid_mapping :

    rotated_pole

    array([9.50443510e-05, 5.12871454e-05, 2.84048370e-05, 1.76260258e-05,
           1.36914660e-05, 1.12402215e-05, 1.32495343e-05, 1.95502644e-05,
           2.93430315e-05, 3.95518593e-05, 4.42765289e-05, 4.11863163e-05,
           3.99953905e-05, 4.61470695e-05, 6.56243590e-05, 1.02707625e-04,
           1.47150780e-04, 1.86406260e-04, 1.95448351e-04, 1.96398987e-04,
           1.87974480e-04, 1.82981234e-04, 1.49367373e-04, 1.03059850e-04,
           7.48802818e-05, 6.40568235e-05, 6.46042950e-05, 6.56105612e-05,
           6.54536432e-05, 6.27221013e-05, 5.55246073e-05, 4.30901493e-05,
           2.85503387e-05, 1.96285762e-05, 1.54400140e-05, 1.21405881e-05,
           1.08775431e-05, 1.19991368e-05, 1.63651271e-05, 2.57284846e-05,
           3.36424820e-05, 5.14614350e-05, 8.65088859e-05, 1.29177819e-04,
           1.62253049e-04, 1.83629215e-04, 2.18722652e-04, 2.32521740e-04,
           1.84020378e-04, 1.58897295e-04, 1.32141259e-04, 1.01039775e-04,
           8.54647682e-05, 7.81932606e-05, 8.54622655e-05, 8.31751199e-05,
           6.08508291e-05, 4.03356020e-05, 2.88819796e-05, 2.45408496e-05,
           2.30376513e-05, 2.21903999e-05, 2.16081323e-05, 2.05030399e-05,
           1.91642192e-05, 1.66044240e-05, 1.43506056e-05, 1.44491685e-05,
           1.52700260e-05, 1.63412276e-05, 1.59698678e-05, 1.53559288e-05,
           1.50377643e-05, 1.50542214e-05, 1.40357646e-05, 1.22557905e-05,
           1.21974014e-05, 1.15465526e-05, 1.04673005e-05, 9.15867187e-06,
    ...
           7.27053840e-05, 1.09718804e-04, 1.19899965e-04, 1.25551833e-04,
           1.16976124e-04, 9.05705632e-05, 7.17977550e-05, 6.43383836e-05,
           5.74943003e-05, 5.47068092e-05, 5.81142249e-05, 6.04854441e-05,
           6.22016725e-05, 6.17544767e-05, 5.54025373e-05, 4.47203786e-05,
           3.33014965e-05, 2.52376392e-05, 2.12805294e-05, 1.94024043e-05,
           1.66496800e-05, 1.42932711e-05, 1.53116509e-05, 1.99877365e-05,
           2.66323328e-05, 3.89808755e-05, 5.56883055e-05, 6.40370333e-05,
           7.93565017e-05, 9.13026485e-05, 5.83375416e-05, 2.77294459e-05,
           2.68155150e-05, 3.70912308e-05, 4.07899004e-05, 2.15035777e-05,
           7.52850135e-06, 3.46254833e-06, 3.43994671e-06, 2.43756369e-06,
           2.11540505e-06, 1.45902417e-06, 1.15341560e-06, 1.07572606e-06,
           1.35157975e-06, 1.87685560e-06, 2.29176929e-06, 2.16917422e-06,
           1.91668475e-06, 2.55326457e-06, 3.11340083e-06, 2.59829936e-06,
           1.40472317e-06, 4.40331195e-07, 7.63446375e-08, 7.88941683e-09,
           3.58540198e-09, 3.04561967e-09, 1.15551466e-08, 6.34113534e-08,
           9.44883470e-08, 3.76358026e-08, 3.76101896e-09, 1.96018676e-09,
           1.65762817e-09, 8.45425682e-17, 1.61598574e-10, 8.95666206e-09,
           9.46372577e-09, 6.04306712e-09, 1.11849614e-09, 3.07398487e-09,
           1.21361724e-08, 6.13119018e-09, 1.23618799e-09, 2.25340621e-10,
           1.02845766e-11, 1.26480086e-13, 1.10137933e-11, 2.56044510e-09])

• Attributes: (41)

  CDI :

  Climate Data Interface version 1.9.8 (https://mpimet.mpg.de/cdi)

  history :

  Mon Mar 29 17:28:32 2021: cdo -O -s -f nc4 -z zip_6 --cmor -cmorlite,/tmp/tmp0ytegbf0.txt /scratch/k/k204230/cdobuqieae0/temp_file_var142.nc4 /scratch/k/k204230/cordex/cordex/EUC-0275/GERICS/ECMWF-ERAINT-GERICS-REMO2015/evaluation/1hr/atmos/1hr/r1i1p1/v20201205/pr/pr_1hr_ECMWF-ERAINT-GERICS-REMO2015_evaluation_r1i1p1_199901010030-199901312330.nc Mon Mar 29 17:28:29 2021: cdo chname,var142,pr /scratch/k/k204230/cdobuqieae0/temp_file_var142.nc /scratch/k/k204230/cdobuqieae0/temp_file_var142.nc4 Mon Mar 29 17:28:24 2021: cdo -f nc4 --cmor splitvar -selindexbox,46,316,46,356 /work/bmx828/integration/team/k204230/hpss/evaluation/exp063001/year1999/e063001e199901/e063001e_c142_199901 /scratch/k/k204230/cdobuqieae0/temp_file_

  Conventions :

  CF-1.6

  tracking_id :

  ed6f10e1-3780-4fba-a454-110790c8bbc3

  creation_date :

  2021-03-29T17:28:32Z

  CDO :

  Climate Data Operators version 1.9.8 (https://mpimet.mpg.de/cdo)

  institution_id :

  GERICS

  institution :

  Helmholtz-Zentrum Geesthacht, Climate Service Center Germany

  contact :

  gerics-cordex@hzg.de

  experiment_id :

  evaluation

  CORDEX_domain :

  EUC-0275

  project_id :

  CORDEX

  initialization_method :

  1

  physics_version :

  1

  frequency :

  sampled hourly

  frequency_id :

  1hr

  modeling_realm :

  atmos

  nominal_resolution :

  3 km

  title :

  GERICS-REMO2015 model output prepared for CORDEX evaluation

  grid :

  0.0275 x 0.0275 degree latitude x longitude

  grid_label :

  gn

  calendar :

  gregorian

  driving_experiment :

  ECMWF-ERAINT, evaluation, r1i1p1

  driving_model_id :

  ECMWF-ERAINT

  driving_experiment_name :

  evaluation

  driving_model_ensemble_member :

  r1i1p1

  rcm_version_id :

  v1

  cmor_version :

  2.9.1

  grid_mapping_name :

  rotated_latitude_longitude

  grid_north_pole_longitude :

  -162.0

  grid_north_pole_latitude :

  39.25

  proj :

  4326

  projection :

  +proj=ob_tran +ellps=WGS84 +a=6378137.0 +o_proj=latlon +o_lon_p=0.0 +o_lat_p=39.25 +lon_0=18.0 +to_meter=111319.4907932736 +no_defs +type=crs

  transform :

  +proj=eqc +ellps=WGS84 +a=6378137.0 +lon_0=0.0 +to_meter=111319.4907932736 +vto_meter=1 +no_defs +type=crs

  filename :

  Time_series_ger-3km_ecmwf-eraint-gerics-remo2015_r1i1p1_evaluation_1hr

  ensemble_member :

  ecmwf-eraint-gerics-remo2015_r1i1p1

  experiment :

  evaluation

  output_freqeuncy :

  1hr

  shape_file :

  /work/ch1187/regiklim-work/k204230/Germany_Regions.geojson

  swift_group :

  ch1187

  path_prefix :

  ProcessedModelData-nukleus-ger-3km-tdps_ts_ps_pr

This dataset is a so called xarray dataset. Xarray is a very powerful library that lets you handle multi-dimensional labeled data arrays. It allows for easy manipulation, visualisation of the underlying data. For example, we can conveniently plot precipitation time-series:

%matplotlib inline
dset['pr'].plot()

[<matplotlib.lines.Line2D at 0x7f4ac77695d0>]

We see the precipitation is given in kg/m^2/s which is according to cf-convention but not very convenient. The data-container offers a method to convert variables units. For example can we convert all units of a given variable to a given target variable, say mm/hour:

data_container.convert("pr", "mm/hour")

This converts all precipitation data arrays to mm/hour. By default the dataset that has been selected by the select method is a so called reference to the data in the data container. Hance changes to either the data-container or the selected dataset will be reflected in both, data-container and selected dataset:

%matplotlib inline
dset['pr'].plot()

[<matplotlib.lines.Line2D at 0x7f4ac6e7a6e0>]

With help of the xarray plotting libary we can inspect the data a little further. For instance comparing the relationship of surface pressure and precipitation:

%matplotlib inline
xr.plot.scatter(dset, 'ps', 'pr')

<matplotlib.collections.PathCollection at 0x7f4ac6d21f90>

Now let’s define a function that calculates the lifted condensation level (lcl):

def lcl(input_data: xr.Dataset) -> xr.Dataset:
    """Calculate the lifted condenstation level with help of metpy.

    Parameters:
    -----------
    input_data:
        Dataset holding data arrays for surface temperature, surface dewpoint and surface pressure

    Returns:
    --------
    Lcl added dataset
    """
    lcl_pres, lcl_temp = metpy.calc.lcl(dset.ps, dset.ts, dset.tdps)
    # Calculate the hight of the LCL in km
    lcl_in_km = metpy.calc.pressure_to_height_std(lcl_pres)
    # Create an xarray DataArray from the lcl height
    lcl_data = xr.DataArray(
        lcl_in_km.magnitude,
        coords=dset.ts.coords,
        dims=dset.ts.dims,
        name="lcl",
        attrs=dset.ts.attrs
    )
    # Attributes from the surface temperature have been added, adjust them to lcl specific ones
    lcl_data.attrs["units"] = "km"
    lcl_data.attrs["standard_name"] = "lcl"
    lcl_data.attrs["long_name"] = "Lifted condensation level"
    lcl_data.attrs.pop("code", "")
    # Add the lcl data to the input dataset
    dset["lcl"] = lcl_data

Now calculate the lcl for all datasets in the data container. We could use the select method or simply loop over the container:

for data in data_container:
    lcl(data)
# Check the variables that are in the data_container
print(data_container.variables)

['tdps', 'ts', 'ps', 'pr', 'lcl']

# Inspect the data set for lcl
data_container.select(by_index=0).lcl

<xarray.DataArray 'lcl' (time: 696)>
array([0.41456444, 0.38369606, 0.355973  , 0.34096036, 0.38573349,
       0.48194181, 0.60859884, 0.75559301, 0.88659971, 0.99915582,
       1.08952576, 1.13127652, 1.13173516, 1.0924129 , 1.03527414,
       0.95968042, 0.87677191, 0.78879979, 0.69924412, 0.6265866 ,
       0.57301881, 0.538067  , 0.51036379, 0.48029791, 0.44640516,
       0.41637347, 0.39702045, 0.3861391 , 0.39138702, 0.41059171,
       0.44804307, 0.50093626, 0.56573122, 0.63705818, 0.69233004,
       0.73383446, 0.75670915, 0.75382099, 0.72976079, 0.689635  ,
       0.63998925, 0.57509006, 0.51027125, 0.47291393, 0.44918392,
       0.43141892, 0.42290889, 0.42053975, 0.42232433, 0.43172892,
       0.44603137, 0.45513331, 0.46913105, 0.5125712 , 0.58794883,
       0.67955442, 0.77308422, 0.84896457, 0.9128196 , 0.9654411 ,
       0.99852701, 1.00744187, 0.99205492, 0.95808374, 0.9063193 ,
       0.8339873 , 0.74800021, 0.68213717, 0.63009184, 0.58461539,
       0.54196965, 0.50393472, 0.47118584, 0.44499188, 0.42515909,
       0.41433054, 0.45054905, 0.54562726, 0.70021679, 0.88734801,
       1.08757643, 1.27669202, 1.44978291, 1.5341508 , 1.56589514,
       1.56285456, 1.52191435, 1.45120444, 1.35485245, 1.23400826,
       1.09329362, 0.98499826, 0.90396661, 0.83911082, 0.78692378,
       0.74712609, 0.71848973, 0.69195525, 0.66645964, 0.64339156,
...
       0.65538609, 0.64113295, 0.63481578, 0.63525513, 0.67630819,
       0.74630743, 0.84794017, 0.9885651 , 1.12710127, 1.23873637,
       1.31604845, 1.36330918, 1.3647003 , 1.32375084, 1.28670149,
       1.23060523, 1.14479746, 1.04018063, 0.92592406, 0.83216509,
       0.77210266, 0.72258264, 0.68019768, 0.64522123, 0.61988448,
       0.60444304, 0.59576371, 0.59215917, 0.60221133, 0.61956134,
       0.64659544, 0.69911054, 0.77411926, 0.84812679, 0.92056375,
       0.98297265, 1.02543236, 1.04405836, 1.04715214, 1.02562276,
       0.97640154, 0.8953255 , 0.8024732 , 0.72806643, 0.67882643,
       0.6447539 , 0.61468327, 0.58728271, 0.55675154, 0.53322448,
       0.51414786, 0.50332888, 0.52412028, 0.58223875, 0.66489977,
       0.81063743, 0.97349592, 1.12970717, 1.26703043, 1.35844563,
       1.41154176, 1.43238367, 1.4181825 , 1.37421005, 1.29709732,
       1.17473155, 1.01938131, 0.90035012, 0.81550632, 0.74073127,
       0.68099528, 0.63726098, 0.60419687, 0.58104336, 0.56347001,
       0.5568276 , 0.62775997, 0.75258238, 0.88919377, 1.12277409,
       1.3494008 , 1.50700854, 1.61980845, 1.6905923 , 1.72002803,
       1.72028893, 1.69239446, 1.63927243, 1.55544829, 1.41552857,
       1.25865598, 1.1577942 , 1.08911537, 1.02139888, 0.95856849,
       0.90222675])
Coordinates:
  * time     (time) datetime64[ns] 2009-06-01T00:30:00 ... 2009-06-29T23:30:00
Attributes:
    standard_name:  lcl
    long_name:      Lifted condensation level
    units:          km
    grid_mapping:   rotated_pole

xarray.DataArray

'lcl'

• time: 696

• 0.4146 0.3837 0.356 0.341 0.3857 ... 1.158 1.089 1.021 0.9586 0.9022

  array([0.41456444, 0.38369606, 0.355973  , 0.34096036, 0.38573349,
         0.48194181, 0.60859884, 0.75559301, 0.88659971, 0.99915582,
         1.08952576, 1.13127652, 1.13173516, 1.0924129 , 1.03527414,
         0.95968042, 0.87677191, 0.78879979, 0.69924412, 0.6265866 ,
         0.57301881, 0.538067  , 0.51036379, 0.48029791, 0.44640516,
         0.41637347, 0.39702045, 0.3861391 , 0.39138702, 0.41059171,
         0.44804307, 0.50093626, 0.56573122, 0.63705818, 0.69233004,
         0.73383446, 0.75670915, 0.75382099, 0.72976079, 0.689635  ,
         0.63998925, 0.57509006, 0.51027125, 0.47291393, 0.44918392,
         0.43141892, 0.42290889, 0.42053975, 0.42232433, 0.43172892,
         0.44603137, 0.45513331, 0.46913105, 0.5125712 , 0.58794883,
         0.67955442, 0.77308422, 0.84896457, 0.9128196 , 0.9654411 ,
         0.99852701, 1.00744187, 0.99205492, 0.95808374, 0.9063193 ,
         0.8339873 , 0.74800021, 0.68213717, 0.63009184, 0.58461539,
         0.54196965, 0.50393472, 0.47118584, 0.44499188, 0.42515909,
         0.41433054, 0.45054905, 0.54562726, 0.70021679, 0.88734801,
         1.08757643, 1.27669202, 1.44978291, 1.5341508 , 1.56589514,
         1.56285456, 1.52191435, 1.45120444, 1.35485245, 1.23400826,
         1.09329362, 0.98499826, 0.90396661, 0.83911082, 0.78692378,
         0.74712609, 0.71848973, 0.69195525, 0.66645964, 0.64339156,
  ...
         0.65538609, 0.64113295, 0.63481578, 0.63525513, 0.67630819,
         0.74630743, 0.84794017, 0.9885651 , 1.12710127, 1.23873637,
         1.31604845, 1.36330918, 1.3647003 , 1.32375084, 1.28670149,
         1.23060523, 1.14479746, 1.04018063, 0.92592406, 0.83216509,
         0.77210266, 0.72258264, 0.68019768, 0.64522123, 0.61988448,
         0.60444304, 0.59576371, 0.59215917, 0.60221133, 0.61956134,
         0.64659544, 0.69911054, 0.77411926, 0.84812679, 0.92056375,
         0.98297265, 1.02543236, 1.04405836, 1.04715214, 1.02562276,
         0.97640154, 0.8953255 , 0.8024732 , 0.72806643, 0.67882643,
         0.6447539 , 0.61468327, 0.58728271, 0.55675154, 0.53322448,
         0.51414786, 0.50332888, 0.52412028, 0.58223875, 0.66489977,
         0.81063743, 0.97349592, 1.12970717, 1.26703043, 1.35844563,
         1.41154176, 1.43238367, 1.4181825 , 1.37421005, 1.29709732,
         1.17473155, 1.01938131, 0.90035012, 0.81550632, 0.74073127,
         0.68099528, 0.63726098, 0.60419687, 0.58104336, 0.56347001,
         0.5568276 , 0.62775997, 0.75258238, 0.88919377, 1.12277409,
         1.3494008 , 1.50700854, 1.61980845, 1.6905923 , 1.72002803,
         1.72028893, 1.69239446, 1.63927243, 1.55544829, 1.41552857,
         1.25865598, 1.1577942 , 1.08911537, 1.02139888, 0.95856849,
         0.90222675])

• Coordinates: (1)
  • time
    (time)
    datetime64[ns]
    2009-06-01T00:30:00 ... 2009-06-...

    standard_name :

    time

    long_name :

    time

    axis :

    T

    array(['2009-06-01T00:30:00.000000000', '2009-06-01T01:30:00.000000000',
           '2009-06-01T02:30:00.000000000', ..., '2009-06-29T21:30:00.000000000',
           '2009-06-29T22:30:00.000000000', '2009-06-29T23:30:00.000000000'],
          dtype='datetime64[ns]')

• Attributes: (4)

  standard_name :

  lcl

  long_name :

  Lifted condensation level

  units :

  km

  grid_mapping :

  rotated_pole

# Convert the lcl's in the data container from km to m
data_container.convert("lcl", "m")
dset = data_container.select(by_name="ecmwf-eraint-gerics-remo2015_r1i1p1")
dset.lcl

<xarray.DataArray 'lcl' (time: 696)>
array([ 414.5644378 ,  383.69605915,  355.97299583,  340.96036309,
        385.73348664,  481.94181487,  608.59884232,  755.5930091 ,
        886.59971001,  999.1558183 , 1089.52576375, 1131.27651758,
       1131.73515841, 1092.41290095, 1035.27413796,  959.6804189 ,
        876.77190869,  788.79979298,  699.24412234,  626.58659954,
        573.01880798,  538.0669983 ,  510.36379247,  480.29791191,
        446.40515614,  416.37347408,  397.02045094,  386.13909966,
        391.38701594,  410.59171114,  448.04307461,  500.93625945,
        565.73122045,  637.05817754,  692.3300373 ,  733.83445809,
        756.70914823,  753.82098729,  729.76079456,  689.63499863,
        639.98924572,  575.09006003,  510.27125066,  472.91392586,
        449.1839199 ,  431.41891788,  422.90888543,  420.53975406,
        422.32433343,  431.72892105,  446.03136641,  455.13330995,
        469.1310462 ,  512.57120465,  587.9488322 ,  679.55441899,
        773.08422263,  848.96456948,  912.81960373,  965.44110408,
        998.52700889, 1007.44187022,  992.05492358,  958.0837391 ,
        906.31929781,  833.98729592,  748.00020826,  682.13717132,
        630.09183837,  584.61538785,  541.96964883,  503.9347241 ,
        471.18583847,  444.99187626,  425.15908537,  414.33053831,
        450.54905204,  545.62725737,  700.21678691,  887.34801284,
...
       1144.79746375, 1040.18062626,  925.9240616 ,  832.16508543,
        772.10265583,  722.58263823,  680.19767623,  645.22122723,
        619.88447604,  604.4430426 ,  595.76370818,  592.1591668 ,
        602.21133413,  619.56133992,  646.5954379 ,  699.11054352,
        774.1192632 ,  848.12679433,  920.56374811,  982.97264855,
       1025.43236038, 1044.05836362, 1047.15214328, 1025.62275534,
        976.40154228,  895.325497  ,  802.47319732,  728.06642892,
        678.82642769,  644.75390177,  614.68326779,  587.28271063,
        556.75154153,  533.22448309,  514.14786255,  503.3288788 ,
        524.12027762,  582.23875364,  664.89977375,  810.6374289 ,
        973.49591538, 1129.70717   , 1267.0304347 , 1358.44563439,
       1411.54176014, 1432.38367427, 1418.18249975, 1374.21005009,
       1297.09732461, 1174.73154963, 1019.38131209,  900.35011577,
        815.50632413,  740.73127188,  680.99527516,  637.26098311,
        604.19687303,  581.04335869,  563.47001387,  556.82760316,
        627.75997263,  752.5823834 ,  889.19376885, 1122.77409186,
       1349.40080369, 1507.00854062, 1619.80845054, 1690.59229688,
       1720.02803088, 1720.28893154, 1692.39446305, 1639.27242536,
       1555.44829345, 1415.52856507, 1258.65597877, 1157.79420204,
       1089.11536672, 1021.39888425,  958.56848661,  902.22674736])
Coordinates:
  * time     (time) datetime64[ns] 2009-06-01T00:30:00 ... 2009-06-29T23:30:00
Attributes:
    standard_name:  lcl
    long_name:      Lifted condensation level
    units:          m
    grid_mapping:   rotated_pole

xarray.DataArray

'lcl'

• time: 696

• 414.6 383.7 356.0 341.0 385.7 ... 1.089e+03 1.021e+03 958.6 902.2

  array([ 414.5644378 ,  383.69605915,  355.97299583,  340.96036309,
          385.73348664,  481.94181487,  608.59884232,  755.5930091 ,
          886.59971001,  999.1558183 , 1089.52576375, 1131.27651758,
         1131.73515841, 1092.41290095, 1035.27413796,  959.6804189 ,
          876.77190869,  788.79979298,  699.24412234,  626.58659954,
          573.01880798,  538.0669983 ,  510.36379247,  480.29791191,
          446.40515614,  416.37347408,  397.02045094,  386.13909966,
          391.38701594,  410.59171114,  448.04307461,  500.93625945,
          565.73122045,  637.05817754,  692.3300373 ,  733.83445809,
          756.70914823,  753.82098729,  729.76079456,  689.63499863,
          639.98924572,  575.09006003,  510.27125066,  472.91392586,
          449.1839199 ,  431.41891788,  422.90888543,  420.53975406,
          422.32433343,  431.72892105,  446.03136641,  455.13330995,
          469.1310462 ,  512.57120465,  587.9488322 ,  679.55441899,
          773.08422263,  848.96456948,  912.81960373,  965.44110408,
          998.52700889, 1007.44187022,  992.05492358,  958.0837391 ,
          906.31929781,  833.98729592,  748.00020826,  682.13717132,
          630.09183837,  584.61538785,  541.96964883,  503.9347241 ,
          471.18583847,  444.99187626,  425.15908537,  414.33053831,
          450.54905204,  545.62725737,  700.21678691,  887.34801284,
  ...
         1144.79746375, 1040.18062626,  925.9240616 ,  832.16508543,
          772.10265583,  722.58263823,  680.19767623,  645.22122723,
          619.88447604,  604.4430426 ,  595.76370818,  592.1591668 ,
          602.21133413,  619.56133992,  646.5954379 ,  699.11054352,
          774.1192632 ,  848.12679433,  920.56374811,  982.97264855,
         1025.43236038, 1044.05836362, 1047.15214328, 1025.62275534,
          976.40154228,  895.325497  ,  802.47319732,  728.06642892,
          678.82642769,  644.75390177,  614.68326779,  587.28271063,
          556.75154153,  533.22448309,  514.14786255,  503.3288788 ,
          524.12027762,  582.23875364,  664.89977375,  810.6374289 ,
          973.49591538, 1129.70717   , 1267.0304347 , 1358.44563439,
         1411.54176014, 1432.38367427, 1418.18249975, 1374.21005009,
         1297.09732461, 1174.73154963, 1019.38131209,  900.35011577,
          815.50632413,  740.73127188,  680.99527516,  637.26098311,
          604.19687303,  581.04335869,  563.47001387,  556.82760316,
          627.75997263,  752.5823834 ,  889.19376885, 1122.77409186,
         1349.40080369, 1507.00854062, 1619.80845054, 1690.59229688,
         1720.02803088, 1720.28893154, 1692.39446305, 1639.27242536,
         1555.44829345, 1415.52856507, 1258.65597877, 1157.79420204,
         1089.11536672, 1021.39888425,  958.56848661,  902.22674736])

• Coordinates: (1)
  • time
    (time)
    datetime64[ns]
    2009-06-01T00:30:00 ... 2009-06-...

    standard_name :

    time

    long_name :

    time

    axis :

    T

    array(['2009-06-01T00:30:00.000000000', '2009-06-01T01:30:00.000000000',
           '2009-06-01T02:30:00.000000000', ..., '2009-06-29T21:30:00.000000000',
           '2009-06-29T22:30:00.000000000', '2009-06-29T23:30:00.000000000'],
          dtype='datetime64[ns]')

• Attributes: (4)

  standard_name :

  lcl

  long_name :

  Lifted condensation level

  units :

  m

  grid_mapping :

  rotated_pole

%matplotlib inline
xr.plot.scatter(dset, "lcl", "pr")

<matplotlib.collections.PathCollection at 0x7f4ac6d9bc40>

We have calculated the lifted condensation level. Now it is time time to save the processed data for future usage. This is done by applying the save method. This method pushes the new dataset into cloud where it is ready for download. The data will be save into zip files. For data files themselves you can choose between the following extensions:

• nc for netcdf4 files,

• h5 for hdf5 files,

• csv for text based csv files.

data_container.save(filetype='csv')

Data was uploaded. Access the data via the following linkto the swift cloud storage.