Preparing data for Met.3D ========================= This section provides examples of preparing data for use with Met.3D. Remapping from spectral grid to regular grid ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The data if available on spectral grid or gaussian grid, needs to be remapped onto a regular lat lon grid, so that it can be analysed using Met.3D. The following code snippet provides a minimal example to perform the remapping. .. code-block:: bash :linenos: #!/bin/bash ##################################################################################### # Minimal examples of basic CDO commands for remapping ERA5 data onto a regular grid # # Link to the Met.3D documentation # https://collaboration.cen.uni-hamburg.de/display/Met3D/Welcome+to+Met.3D # # Link to the cdo documentation # https://code.mpimet.mpg.de/projects/cdo/wiki/Cdo#Documentation # # Link to a cdo reference sheet # https://code.mpimet.mpg.de/projects/cdo/embedded/cdo_refcard.pdf # ##################################################################################### ########### # define path to the folder storing the ERA5 data as provided by the ECMWF # (GRIB data with spatial coordinate systems "Gaussian Grid" or "Spherical Harmonics") DATAFOLDER=/path/to/ERA5-data-folder/ ########## # define path to the folder for storing the remapped ERA5 data-files POSTPROCESSEDDATAFOLDER=/path/to/Results-Folder/ mkdir -p ${POSTPROCESSEDDATAFOLDER} ########## # define path to a file characterizing the grid to which the ERA5 data will be remapped TARGETGRIDFILE=/path/to/targetgridfile.txt # See below for an examplar gridFile for remapping with CDO to a regular lat-lon grid : # --> "cdo_gridFile_regular_LatLon_minimalExample.txt" ########## # define input and output data files INFILE=/path/to/ERA5_Data_File OUTFILE=/path/to/ERA5_Data_on_regular_grid ########## # bilinear remapping from reduced Gaussian grid to regular Lat-Lon grid # and conversion from GRIB to NETCDF cdo -f nc4 -remapbil,${TARGETGRIDFILE} -setgridtype,regular ${OUTFILE} ${INFILE} ########## # bilinear remapping from spectral grid to Gaussian grid to regular Lat-Lon grid cdo -f nc4 -remapbil,${TARGETGRIDFILE} -sp2gpl ${INFILE} ${OUTFILE} The contents of the ``cdo_gridFile_regular_LatLon_minimalExample.txt`` in this example are as below: .. code-block:: :linenos: gridtype = lonlat gridsize = 90381 xsize = 641 ysize = 141 xfirst = -80 xinc = 0.25 yfirst = 85 yinc = -0.25 Remapping from radar grid to regular lat lon grid ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If you want to analyse radar data with actors besides the radar actor or raycaster actor, it needs to be remapped to a regular lat lon grid. When you add a new variable to an actor and have loaded a radar dataset, you can select a radar variable with "Pressure Levels" instead of "Radar Elevation" as the vertical dimension. This causes the radar data to be remapped to a grid that is regular in the horizontal and has pressure levels as its vertical dimension. After adding the variable, you can modify its **regrid settings**: * Change the vertical grid between equidistant pressure levels, equidistant height levels and user defined pressure levels. * Change the number of equidistant pressure/height levels. * For the horizontal grid dimensions, you can change the number of grid points or the distance between grid points in metres. Remapping icosahedral ICON data to regular grid ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Currently though Met3D doesn't support the native unstructured grid of ICON model, it is still possible to remap the grid onto a regular lat lon grid format supported by Met3D. The following are the steps that outline this procedure: * The 'reference time' of the 'time' variable should be renamed to a valid units as per CF convention. For example, below to set to 'hours' since '2020-02-01,00:00:00', the CDO command would be ``cdo -setreftime,2020-02-01,00:00:00,hours in_file out_file`` The following example shows the time attributes in the original data file .. code-block:: :linenos: netcdf output_DOM01_PL_0001 { dimensions: time = UNLIMITED ; // (1 currently) lon = 360 ; lat = 71 ; plev = 52 ; variables: double time(time) ; time:standard_name = "time" ; time:units = "day as %Y%m%d.%f" ; time:calendar = "proleptic_gregorian" ; time:axis = "T" ; float lon(lon) ; lon:standard_name = "longitude" ; lon:long_name = "longitude" ; lon:units = "degrees_east" ; lon:axis = "X" ; float lat(lat) ; lat:standard_name = "latitude" ; lat:long_name = "latitude" ; lat:units = "degrees_north" ; lat:axis = "Y" ; double plev(plev) ; plev:standard_name = "air_pressure" ; plev:long_name = "pressure" ; plev:units = "Pa" ; plev:positive = "down" ; plev:axis = "Z" ; float u(time, plev, lat, lon) ; u:standard_name = "eastward_wind" ; u:long_name = "Zonal wind" ; u:units = "m s-1" ; u:param = "2.2.0" ; float v(time, plev, lat, lon) ; v:standard_name = "northward_wind" ; v:long_name = "Meridional wind" ; v:units = "m s-1" ; v:param = "3.2.0" ; The modified attributes for the time variable should appear as below .. code-block:: :linenos: netcdf output_DOM01_PL_0001 { dimensions: time = UNLIMITED ; // (1 currently) lon = 360 ; lat = 71 ; plev = 52 ; variables: double time(time) ; time:standard_name = "time" ; time:units = "hours since 2020-2-1 00:00:00" ; time:calendar = "proleptic_gregorian" ; time:axis = "T" ; float lon(lon) ; lon:standard_name = "longitude" ; lon:long_name = "longitude" ; lon:units = "degrees_east" ; lon:axis = "X" ; float lat(lat) ; lat:standard_name = "latitude" ; lat:long_name = "latitude" ; lat:units = "degrees_north" ; lat:axis = "Y" ; double plev(plev) ; plev:standard_name = "air_pressure" ; plev:long_name = "pressure" ; plev:units = "Pa" ; plev:positive = "down" ; plev:axis = "Z" ; float u(time, plev, lat, lon) ; u:standard_name = "eastward_wind" ; u:long_name = "Zonal wind" ; u:units = "m s-1" ; u:param = "2.2.0" ; float v(time, plev, lat, lon) ; v:standard_name = "northward_wind" ; v:long_name = "Meridional wind" ; v:units = "m s-1" ; v:param = "3.2.0" ; * The files, if existing in different directories for different ensemble members, can be accessed by Met3D data pipeline, by using wild card characters as shown in the Figure... **TODO: Add Figure** * Also, if the user wants to create a single file containing all the ensemble members and all time steps (say for a given variable), then the following preprocessing steps need to be done: #. Concatenate all time steps of given member using ``ncrcat -n nsteps,4,1 ${idir}/output_DOM01_PL_0001.nc ${odir}/output_DOM01_PL_all.nc`` where * ``nsteps``: number of time steps. * ``4``: the numeric suffix digit position that is to be incremented, here '**0001**' in the filename '**output_DOM01_PL_0001.nc**'. * ``1``: increment. * ``idir``: Input data files directory. * ``odir``: Ouput data files directory. #. Concatenate all ensemble member's all timesteps file, generated in the above step into single file using ``ncecat -3 -u ensemble ${ifiles} ${odir}/output_DOM01_PL_ens.nc`` where * ``3``: netcdf3 format. * ``7``: netcdf4 classic. * ``u``: name of new dimension, here 'ensemble'. * ``odir``: Ouput data files directory and 'output_DOM01_PL_ens.nc' is the name of the output file. #. The record dimension of the file needs to changed to the newly created 'ensemble' dimension using ``ncpdq -a time,ensemble ${idir}/output_DOM01_PL_ens.nc ${odir}/output_DOM01_PL_final.nc`` where * ``-a time,ensemble``: Indication that 'time' dimension needs to be exchange position with 'ensemble' dimension, in the order of the dimensions. * ``idir``: Input data files directory and 'output_DOM01_PL_ens.nc' is the file with all time steps for all ensemble members. * ``odir``: Ouput data files directory and 'output_DOM01_PL_final.nc' is the name of the output file. Creating stereographic data from regular lat lon data ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Met3D supports various projection types that are available in the 'proj' library. If the user intends to explore these features, then the regular lat lon grid first needs to be regridded onto the appropriate projection and then visualized in Met3D. This remapping can be done as outlined below, for example from regular lat lon to stereographic using CDO command .. code-block:: bash cdo -P 4 -O remapbil,${TARGETGRIDFILE} ${INFILE} ${OUTFILE} where * ``-P 4``: The number of pthreads to speed up the processing (here **4**). * ``remapbil``: CDO operator to do remapping to a particular grid defined in the **TARGETGRIDFILE**. * ``TARGETGRIDFILE``: A text file containing the target grid description , for example: .. code-block:: :linenos: gridtype = projection gridsize = 25600 xsize = 160 ysize = 160 xname = xc xlongname = "x coordinate of projection (eastings)" xunits = "km" yname = yc ylongname = "y coordinate of projection (northings)" yunits = "km" xfirst = -8000 xinc = 100 yfirst = 8000 yinc = -100 grid_mapping = Polar_Stereographic_Grid grid_mapping_name = polar_stereographic straight_vertical_longitude_from_pole = 0.f latitude_of_projection_origin = 90.f standard_parallel = 70.f false_easting = 0.f false_northing = 0.f semi_major_axis = 6378273.f semi_minor_axis = 6356890.f proj4_string = "+proj=stere +a=6378273 +b=6356889.44891 +lat_0=90 +lat_ts=70 +lon_0=0" # distances in km and grid res from here: https://nsidc.org/data/polar-stereo/ps_grids.html # other example grid, see: https://code.mpimet.mpg.de/projects/cdo/embedded/index.html#x1-130001.3