Variable pipeline
=================

Each NWP data source exposed to actors is the end of a fixed processing chain
assembled at startup from ``pipelineconfiguration.cpp``. The chain consists of
several *pipeline stages*, each implemented as a processing data source. A
stage that has nothing to do for a given request passes it upstream unchanged
(pass-through), so only explicitly enabled stages perform any computation.


Pipeline for direct variables
------------------------------

For variables that are read directly from disk the chain is::

    Reader (NetCDF, GRIB)
      -> Smooth filter
      -> Partial deriv filter
      -> Ensemble filter
      -> Vertical regridder

The smooth and partial derivative filters are opt-in: the actor variable's
property panel does not insert the corresponding request key when the filter is
disabled, so the memory manager treats the request as if that stage
does not exist. The ensemble filter is always present and handles operations
such as ensemble mean, standard deviation, and probability computations.

The assembled pipeline is registered in the system under the pipeline name
configured in the dataset configuration file. This name is the actor variable's
data source.


Pipeline for derived variables
--------------------------------

A parallel chain handles variables that are computed from other variables
(e.g. wind speed from U and V). ``MDerivedMetVarsDataSource`` is inserted
between the reader and the rest of the stages::

    Reader
      -> MDerivedMetVarsDataSource
      -> Smooth filter
      -> Partial deriv filter
      -> Ensemble filter
      -> Vertical regridder

``MDerivedMetVarsDataSource`` has a single upstream source and internally
requests all required input fields from that same source. Which input variable
names map to which abstract role (e.g. which NetCDF variable name provides
``eastward_wind``) is configured via the ``inputVarsForDerivedVars`` entries in
the pipeline configuration file.

The derived pipeline is registered under ``<pipeline name> derived``
(e.g. ``ECMWF-ENS derived``) and appears as a separate entry in the data
source selector.


Smoothing stage setting
-----------------------

A global user setting controls whether smoothing is applied before or after the
derived variable computation. This affects where the smooth filter is wired in
the derived chain.

**Smooth before derived variable computation**::

    Direct:   Reader -> Smooth -> PartialDeriv -> Ensemble
    Derived:  Reader -> Smooth -> DerivedVars -> PartialDeriv -> Ensemble

Both chains share a single smooth filter instance. Input fields such as U and V
are smoothed before wind speed (or any other derived quantity) is computed.

**Smooth after derived variable computation**::

    Direct:   Reader -> Smooth -> PartialDeriv -> Ensemble
    Derived:  Reader -> DerivedVars -> Smooth -> PartialDeriv -> Ensemble

Two separate smooth filter instances are used. Input fields are passed raw to
the derived vars source; the derived result is then smoothed.

The setting is controlled via the smoothing stage option in the Met.3D user
configuration.


Difference data source
-----------------------

When an actor variable is configured to show the field difference between two
data sources, a ``MDifferenceDataSource`` is inserted in front of the normal
pipeline. It connects to two separate pipeline endpoints and subtracts one
output from the other before passing the result downstream. See the user manual
section on computed difference fields for details on how to configure this.