Gridded data (horizontally regular)

Met.3D supports gridded, structured data with a regular longitude-latitude grid in the horizontal, and either levels of constant pressure (in the following “pressure levels”) or hybrid sigma-pressure levels (as used, e.g., by the ECMWF integrated forecast system) in the vertical.

Internally, a single 2D or 3D scalar data field (i.e., one forecast variable, one time step, one ensemble member) is treated as the smallest data entity. A single data field is stored in a class derived from MStructuredGrid, these objects store all contextual information required to visualise the field (including latitude and longitude coordinates, pressure or hybrid level, time step and additional metadata). Example of derived classes are MRegularLonLatGrid for 2D fields, MRegularLonLatStructuredPressureGrid for 3D fields on pressure levels, MLonLatHybridSigmaPressureGrid for 3D fields on hybrid sigma-pressure levels. Time series and ensemble sets are composed of these basic data field entities.

Gridded data in NetCDF format

Gridded, structured data can be read from NetCDF files that follow the Climate and Forecast (CF) Metadata Conventions. For more information on how to load these, see ../data_handling/loading_data.

Note

Forecast variables, time steps and ensemble members can be arbitrarily distributed over the files that match the file filter. You can store all ensemble members in one file but have different files for each time step, or vice versa. Or everything can be stored in a single file.

Examples of how NetCDF files need to be structured

The following example contains a NetCDF header of a file containing an ensemble forecast on pressure levels.

 1netcdf somegriddeddata.pl {
 2dimensions:
 3    lon = 101 ;
 4    lat = 41 ;
 5    isobaric = 12 ;
 6    time = 1 ;
 7    ens0 = 51 ;
 8variables:
 9    float lat(lat) ;
10            lat:units = "degrees_north" ;
11    float lon(lon) ;
12            lon:units = "degrees_east" ;
13    float isobaric(isobaric) ;
14            isobaric:units = "hPa" ;
15            isobaric:long_name = "Isobaric surface" ;
16            isobaric:positive = "down" ;
17    int time(time) ;
18            time:units = "Hour since 2012-10-15T00:00:00.000Z" ;
19            time:standard_name = "time" ;
20    int ens0(ens0) ;
21            ens0:standard_name = "ensemble_member_id" ;
22
23    float Geopotential_isobaric(time, ens0, isobaric, lat, lon) ;
24            Geopotential_isobaric:long_name = "Geopotential @ Isobaric surface" ;
25            Geopotential_isobaric:units = "m2.s-2" ;
26    float Temperature_isobaric(time, ens0, isobaric, lat, lon) ;
27            Temperature_isobaric:long_name = "Temperature @ Isobaric surface" ;
28            Temperature_isobaric:units = "K" ;
29
30    ...
31}

Note the following:

Important

The time encoded in the units attribute of the time dimension is used as the forecast base/initialisation time. If you are not using forecast data (e.g. if you are using reanalysis data), Met.3D requires you to either use the same reference time in all files of your dataset, or to have the reference time equal the valid time (i.e. lead time = 0) - this essentially means that you are using one file per time step.

  • The ensemble dimension is optional. If you don’t have an ensemble dimension, the variables should have the dimensions (time, isobaric, lat, lon).

Important

The ensemble dimension is currently not specified in the CF conventions. Met.3D simply searches for a variable with a standard_name attribute set to ensemble_member_id. For now, the _CoordinateAxisType used by the netcdf-java library (_CoordinateAxisType = "Ensemble") is also acceptable. (See the Met.3D source code method NcCFVar::getEnsembleVar() in nccfvar.cpp).

The following example contains a NetCDF header of a file containing an ensemble forecast on hybrid sigma-pressure levels; the required keywords of the vertical variable are different; cf. Appendix D of the CF-conventions.

 1netcdf somegriddeddata.ml {
 2dimensions:
 3    lon = 101 ;
 4    lat = 41 ;
 5    hybrid = 62 ;
 6    time = 1 ;
 7    ens0 = 51 ;
 8variables:
 9    float lat(lat) ;
10            lat:units = "degrees_north" ;
11    float lon(lon) ;
12            lon:units = "degrees_east" ;
13    float hybrid(hybrid) ;
14            hybrid:units = "sigma" ;
15            hybrid:long_name = "Hybrid level" ;
16            hybrid:positive = "down" ;
17            hybrid:standard_name = "atmosphere_hybrid_sigma_pressure_coordinate" ;
18            hybrid:formula = "p(time,level,lat,lon) = ap(level) + b(level)*ps(time,lat,lon)" ;
19            hybrid:formula_terms = "ap: hyam b: hybm ps: Surface_pressure_surface" ;
20    int time(time) ;
21            time:units = "Hour since 2012-10-15T12:00:00.000Z" ;
22            time:standard_name = "time" ;
23    int ens0(ens0) ;
24            ens0:_CoordinateAxisType = "Ensemble" ;
25
26    double hyam(hybrid) ;
27            hyam:long_name = "hybrid A coefficient at layer midpoints" ;
28            hyam:units = "Pa" ;
29    double hybm(hybrid) ;
30            hybm:long_name = "hybrid B coefficient at layer midpoints" ;
31            hybm:units = "1" ;
32
33    float Temperature_hybrid(time, ens0, hybrid, lat, lon) ;
34            Temperature_hybrid:long_name = "Temperature @ Hybrid level" ;
35            Temperature_hybrid:units = "K" ;
36    float Specific_humidity_hybrid(time, ens0, hybrid, lat, lon) ;
37            Specific_humidity_hybrid:long_name = "Specific humidity @ Hybrid level" ;
38            Specific_humidity_hybrid:units = "kg/kg" ;
39
40    ...
41}

Important

Met.3D requires the surface pressure field as well as the ak/bk coefficients to reconstruct the 3D pressure field. The names of these variables are specified in the “hybrid:formula_terms” attribute. Make sure that this attribute lists the correct variable names. A mismatch is a common source for errors - Met.3D will not be able to find the required fields!

Gridded data in GRIB format

Met.3D provides support for GRIB files written by ECMWF’s ecCodes library (as output, e.g. by the ECMWF MARS archive or written by Metview).

In the GUI, where you create or load dataset configurations, simply change the file format to GRIB. Alternatively you can also edit the dataset configuration flile directly and change the fileFormat entry to GRIB.

Note the following:

  • GRIB messages may be arbitrarily distributed over different files that match the specified fileFilter.

  • Support is currently only provided for a horizontally regular longitude/latitude grid (GRIB gridType = regular_ll) and either pressure (GRIB typeOfLevel = isobaricInhPa) or hybrid sigma-pressure levels (GRIB typeOfLevel = hybrid) in the vertical.

  • Hybrid sigma-pressure model levels additionally require the surface pressure field (GRIB shortName = sp) or its logarithm (GRIB shortName = lnsp) to be present in the dataset.

  • Analysis (GRIB dataType = an), forecast (GRIB dataType = fc) and perturbed/control (i.e., ensemble forecast; GRIB dataType = pf/cf) messages are interpreted. If your dataset contains more than one of these types, it is recommended to load these as separate datasets.

  • For 3D fields, a consistent consecutive list of vertical levels must be present for all time steps and ensemble members of a dataset. Levels need not be complete (i.e., there can be missing levels at the top or bottom), but the same levels need to be provided for all data fields in a dataset.

  • For a given dataset, all GRIB messages must have the same horizontal extent.

Important

Unlike NetCDF files, GRIB files do not contain global headers that summarize the information contained in the file. Hence, to know which data is stored in a file the entire file needs to be read. This, of course, is inefficient when the file is accessed multiple times from different Met.3D sessions. Met.3D hence creates index files that store such a “global header” when first accessing GRIB files. You will find these files along with the GRIB files in the directory in which the GRIB files are stored. Note that your user ID hence requires write access to the directory in which the GRIB files are stored. If you don’t have write access, we recommend placing symlinks to the files in a directory to which you have access and in which the index files can be stored. When you are first loading GRIB files, creation of index files can take some time!