Preparing data for Met.3D

This section provides examples of preparing data for use with Met.3D.

Remapping from spectral grid to regular grid

The data if available on spectral grid or gaussian grid, needs to be remapped onto a regular lat lon grid, so that it can be analysed using Met.3D. The following code snippet provides a minimal example to perform the remapping.

 1#!/bin/bash
 2
 3#####################################################################################
 4# Minimal examples of basic CDO commands for remapping ERA5 data onto a regular grid
 5#
 6# Link to the Met.3D documentation
 7# https://collaboration.cen.uni-hamburg.de/display/Met3D/Welcome+to+Met.3D
 8#
 9# Link to the cdo documentation
10# https://code.mpimet.mpg.de/projects/cdo/wiki/Cdo#Documentation
11#
12# Link to a cdo reference sheet
13# https://code.mpimet.mpg.de/projects/cdo/embedded/cdo_refcard.pdf
14#
15#####################################################################################
16
17
18###########
19# define path to the folder storing the ERA5 data as provided by the ECMWF
20# (GRIB data with spatial coordinate systems "Gaussian Grid" or "Spherical Harmonics")
21DATAFOLDER=/path/to/ERA5-data-folder/
22
23
24##########
25# define path to the folder for storing the remapped ERA5 data-files
26POSTPROCESSEDDATAFOLDER=/path/to/Results-Folder/
27mkdir -p ${POSTPROCESSEDDATAFOLDER}
28
29
30##########
31# define path to a file characterizing the grid to which the ERA5 data will be remapped
32TARGETGRIDFILE=/path/to/targetgridfile.txt
33
34# See below for an examplar gridFile for remapping with CDO to a regular lat-lon grid :
35# --> "cdo_gridFile_regular_LatLon_minimalExample.txt"
36
37
38##########
39# define input and output data files
40INFILE=/path/to/ERA5_Data_File
41OUTFILE=/path/to/ERA5_Data_on_regular_grid
42
43
44##########
45# bilinear remapping from reduced Gaussian grid to regular Lat-Lon grid
46# and conversion from GRIB to NETCDF
47cdo -f nc4 -remapbil,${TARGETGRIDFILE} -setgridtype,regular ${OUTFILE} ${INFILE}
48
49
50##########
51# bilinear remapping from spectral grid to Gaussian grid to regular Lat-Lon grid
52cdo -f nc4 -remapbil,${TARGETGRIDFILE} -sp2gpl ${INFILE} ${OUTFILE}

The contents of the cdo_gridFile_regular_LatLon_minimalExample.txt in this example are as below:

1gridtype = lonlat
2gridsize = 90381
3xsize = 641
4ysize = 141
5xfirst = -80
6xinc = 0.25
7yfirst = 85
8yinc = -0.25

Remapping from radar grid to regular lat lon grid

If you want to analyse radar data with actors besides the radar actor or raycaster actor, it needs to be remapped to a regular lat lon grid.

When you add a new variable to an actor and have loaded a radar dataset, you can select a radar variable with “Pressure Levels” instead of “Radar Elevation” as the vertical dimension. This causes the radar data to be remapped to a grid that is regular in the horizontal and has pressure levels as its vertical dimension.

After adding the variable, you can modify its regrid settings:

  • Change the vertical grid between equidistant pressure levels, equidistant height levels and user defined pressure levels.

  • Change the number of equidistant pressure/height levels.

  • For the horizontal grid dimensions, you can change the number of grid points or the distance between grid points in metres.

Remapping icosahedral ICON data to regular grid

Currently though Met3D doesn’t support the native unstructured grid of ICON model, it is still possible to remap the grid onto a regular lat lon grid format supported by Met3D.

The following are the steps that outline this procedure:

  • The ‘reference time’ of the ‘time’ variable should be renamed to a valid units as per CF convention. For example, below to set to ‘hours’ since ‘2020-02-01,00:00:00’, the CDO command would be cdo -setreftime,2020-02-01,00:00:00,hours in_file out_file

The following example shows the time attributes in the original data file

 1netcdf output_DOM01_PL_0001 {
 2dimensions:
 3        time = UNLIMITED ; // (1 currently)
 4        lon = 360 ;
 5        lat = 71 ;
 6        plev = 52 ;
 7variables:
 8        double time(time) ;
 9                time:standard_name = "time" ;
10                time:units = "day as %Y%m%d.%f" ;
11                time:calendar = "proleptic_gregorian" ;
12                time:axis = "T" ;
13        float lon(lon) ;
14                lon:standard_name = "longitude" ;
15                lon:long_name = "longitude" ;
16                lon:units = "degrees_east" ;
17                lon:axis = "X" ;
18        float lat(lat) ;
19                lat:standard_name = "latitude" ;
20                lat:long_name = "latitude" ;
21                lat:units = "degrees_north" ;
22                lat:axis = "Y" ;
23        double plev(plev) ;
24                plev:standard_name = "air_pressure" ;
25                plev:long_name = "pressure" ;
26                plev:units = "Pa" ;
27                plev:positive = "down" ;
28                plev:axis = "Z" ;
29        float u(time, plev, lat, lon) ;
30                u:standard_name = "eastward_wind" ;
31                u:long_name = "Zonal wind" ;
32                u:units = "m s-1" ;
33                u:param = "2.2.0" ;
34        float v(time, plev, lat, lon) ;
35                v:standard_name = "northward_wind" ;
36                v:long_name = "Meridional wind" ;
37                v:units = "m s-1" ;
38                v:param = "3.2.0" ;

The modified attributes for the time variable should appear as below

 1netcdf output_DOM01_PL_0001 {
 2dimensions:
 3        time = UNLIMITED ; // (1 currently)
 4        lon = 360 ;
 5        lat = 71 ;
 6        plev = 52 ;
 7variables:
 8        double time(time) ;
 9                time:standard_name = "time" ;
10                time:units = "hours since 2020-2-1 00:00:00" ;
11                time:calendar = "proleptic_gregorian" ;
12                time:axis = "T" ;
13        float lon(lon) ;
14                lon:standard_name = "longitude" ;
15                lon:long_name = "longitude" ;
16                lon:units = "degrees_east" ;
17                lon:axis = "X" ;
18        float lat(lat) ;
19                lat:standard_name = "latitude" ;
20                lat:long_name = "latitude" ;
21                lat:units = "degrees_north" ;
22                lat:axis = "Y" ;
23        double plev(plev) ;
24                plev:standard_name = "air_pressure" ;
25                plev:long_name = "pressure" ;
26                plev:units = "Pa" ;
27                plev:positive = "down" ;
28                plev:axis = "Z" ;
29        float u(time, plev, lat, lon) ;
30                u:standard_name = "eastward_wind" ;
31                u:long_name = "Zonal wind" ;
32                u:units = "m s-1" ;
33                u:param = "2.2.0" ;
34        float v(time, plev, lat, lon) ;
35                v:standard_name = "northward_wind" ;
36                v:long_name = "Meridional wind" ;
37                v:units = "m s-1" ;
38                v:param = "3.2.0" ;

  • The files, if existing in different directories for different ensemble members, can be accessed by Met3D data pipeline, by using wild card characters as shown in the Figure…

TODO: Add Figure

  • Also, if the user wants to create a single file containing all the ensemble members and all time steps (say for a given variable), then the following preprocessing steps need to be done:

    1. Concatenate all time steps of given member using

      ncrcat -n nsteps,4,1 ${idir}/output_DOM01_PL_0001.nc  ${odir}/output_DOM01_PL_all.nc where

      • nsteps: number of time steps.

      • 4: the numeric suffix digit position that is to be incremented, here ‘0001’ in the filename ‘output_DOM01_PL_0001.nc’.

      • 1: increment.

      • idir: Input data files directory.

      • odir: Ouput data files directory.

    2. Concatenate all ensemble member’s all timesteps file, generated in the above step into single file using ncecat -3 -u ensemble ${ifiles} ${odir}/output_DOM01_PL_ens.nc where

      • 3: netcdf3 format.

      • 7: netcdf4 classic.

      • u: name of new dimension, here ‘ensemble’.

      • odir: Ouput data files directory and ‘output_DOM01_PL_ens.nc’ is the name of the output file.

    3. The record dimension of the file needs to changed to the newly created ‘ensemble’ dimension using ncpdq -a time,ensemble ${idir}/output_DOM01_PL_ens.nc ${odir}/output_DOM01_PL_final.nc where

      • -a time,ensemble: Indication that ‘time’ dimension needs to be exchange position with ‘ensemble’ dimension, in the order of the dimensions.

      • idir: Input data files directory and ‘output_DOM01_PL_ens.nc’ is the file with all time steps for all ensemble members.

      • odir: Ouput data files directory and ‘output_DOM01_PL_final.nc’ is the name of the output file.

Creating stereographic data from regular lat lon data

Met3D supports various projection types that are available in the ‘proj’ library. If the user intends to explore these features, then the regular lat lon grid first needs to be regridded onto the appropriate projection and then visualized in Met3D.

This remapping can be done as outlined below, for example from regular lat lon to stereographic using CDO command

cdo -P 4 -O remapbil,${TARGETGRIDFILE} ${INFILE} ${OUTFILE}

where

  • -P 4: The number of pthreads to speed up the processing (here 4).

  • remapbil: CDO operator to do remapping to a particular grid defined in the TARGETGRIDFILE.

  • TARGETGRIDFILE: A text file containing the target grid description , for example:

 1gridtype = projection
 2gridsize = 25600
 3xsize = 160
 4ysize = 160
 5xname = xc
 6xlongname = "x coordinate of projection (eastings)"
 7xunits = "km"
 8yname = yc
 9ylongname = "y coordinate of projection (northings)"
10yunits = "km"
11xfirst = -8000
12xinc = 100
13yfirst = 8000
14yinc = -100
15grid_mapping = Polar_Stereographic_Grid
16grid_mapping_name = polar_stereographic
17straight_vertical_longitude_from_pole = 0.f
18latitude_of_projection_origin = 90.f
19standard_parallel = 70.f
20false_easting = 0.f
21false_northing = 0.f
22semi_major_axis = 6378273.f
23semi_minor_axis = 6356890.f
24proj4_string = "+proj=stere +a=6378273 +b=6356889.44891 +lat_0=90 +lat_ts=70 +lon_0=0"
25
26# distances in km and grid res from here: https://nsidc.org/data/polar-stereo/ps_grids.html
27# other example grid, see: https://code.mpimet.mpg.de/projects/cdo/embedded/index.html#x1-130001.3