.. _adding_derived_variables:

Adding a new derived variable
==============================

Every derived variable is implemented as a subclass of ``MDerivedDataFieldProcessor``.
Each processor declares the CF standard name of the output variable,
the list of required input variable names (optionally qualified with a level type),
and a ``compute()`` method that fills the output grid.

All processors are registered in ``MDerivedMetVarsDataSource::MDerivedMetVarsDataSource()``
in ``src/data/derivedvars/derivedmetvarsdatasource.cpp``.

For a list of existing derived variables and user-facing configuration,
see :doc:`/03_user_manual/data_handling/derived_vars`.


Adding a C++ derived variable
------------------------------

1. Subclass ``MDerivedDataFieldProcessor`` in ``src/data/derivedvars/derivedmetvars_standard.h/.cpp``
   (or a new file):

   .. code-block:: c++

       class MMyNewVariableProcessor : public MDerivedDataFieldProcessor
       {
       public:
           MMyNewVariableProcessor();
           void compute(const QList<MDataLease<MStructuredGrid>>& inputGrids,
                        MStructuredGrid *derivedGrid) override;
       };

2. Implement the constructor to declare the standard name and required inputs:

   .. code-block:: c++

       MMyNewVariableProcessor::MMyNewVariableProcessor()
           : MDerivedDataFieldProcessor(
                 "my_new_variable",
                 QStringList() << "air_temperature" << "specific_humidity")
       {}

   To restrict to a specific level type, append a qualifier to the input name
   and pass allowed output level types as a third argument:

   .. code-block:: c++

       MMyNewVariableProcessor::MMyNewVariableProcessor()
           : MDerivedDataFieldProcessor(
                 "my_new_variable",
                 QStringList() << "eastward_wind/HYBRID_SIGMA_PRESSURE_3D"
                               << "surface_air_pressure/SURFACE_2D",
                 {HYBRID_SIGMA_PRESSURE_3D})
       {}

3. Implement ``compute()``. The ``inputGrids`` list matches the declared input order.
   Write results to ``derivedGrid``:

   .. code-block:: c++

       void MMyNewVariableProcessor::compute(
               const QList<MDataLease<MStructuredGrid>>& inputGrids,
               MStructuredGrid *derivedGrid)
       {
           const auto& tempGrid = inputGrids.at(0);
           const auto& humGrid = inputGrids.at(1);
           for (int k = 0; k < derivedGrid->getNumLevels(); ++k)
           {
               for (int j = 0; j < derivedGrid->getNumLats(); ++j)
               {
                   for (int i = 0; i < derivedGrid->getNumLons(); ++i)
                   {
                       float value = 0.f; /* your computation here... */
                       derivedGrid->setValue(k, j, i, value);
                    }
                }
            }
       }

4. Register the processor in ``MDerivedMetVarsDataSource::MDerivedMetVarsDataSource()``
   in ``derivedmetvarsdatasource.cpp``:

   .. code-block:: c++

       registerDerivedDataFieldProcessor(new MMyNewVariableProcessor());


Adding a Python-based derived variable
----------------------------------------

Python-derived variables call MetPy (or arbitrary Python code) through Met.3D's Python
interface. Data is exchanged as an ``xarray.Dataset`` via an in-memory NetCDF buffer.

1. **Add an endpoint type.** In ``src/data/derivedvars/pythonderivedprocessor.h``,
   add a value to the ``EndpointType`` enum before ``INVALID``:

   .. code-block:: c++

       enum EndpointType
       {
           METPY_PV_BAROCLINIC = 0,
           // ... existing values ...
           MY_NEW_ENDPOINT = N,  // next available index
           INVALID
       };

2. **Expose the value to Python.** In ``src/system/mpyinterfacebindings.cpp``,
   add a ``.value()`` line to the pybind11 enum block:

   .. code-block:: c++

       .value("MY_NEW_ENDPOINT", PyEndpoint::MY_NEW_ENDPOINT)

3. **Register the processor.** In ``MPythonDerivedProcessor::registerProcessorsTo()``
   in ``pythonderivedprocessor.cpp``:

   .. code-block:: c++

       auto myProcessor = new MPythonDerivedProcessor(
           "my_new_variable_metpy",
           {"air_pressure", "air_temperature"},  // required inputs, in order
           PyEndpoint::MY_NEW_ENDPOINT);
       dataSource->registerDerivedDataFieldProcessor(myProcessor);

4. **Add the Python function.** In ``src/python/interface/python_endpoints.py``:

   .. code-block:: python

       def my_new_variable(dataset, in_grid_names, out_grid_name):
           import metpy.calc as mpcalc
           from metpy.units import units
           p = dataset[in_grid_names[0]] * units('Pa')
           t = dataset[in_grid_names[1]] * units('K')
           dataset[out_grid_name] = mpcalc.some_function(p, t)
           return dataset

5. **Dispatch the function.** In ``invoke_python_endpoint()`` in ``python_endpoints.py``:

   .. code-block:: python

       elif func == derived_var_bindings.EndpointType.MY_NEW_ENDPOINT:
           return my_new_variable(dataset, in_grid_names, out_grid_name)

.. note::
    If you have implemented a derived variable that you think would be useful to other
    Met.3D users, we would be happy to include it in the main codebase.
    Please consider submitting a `merge request on GitLab`_.

.. _`merge request on GitLab`: https://gitlab.com/wxmetvis/met.3d/-/merge_requests