Data items
==========

A *data item* is the unit of data produced by a data source and stored in
the memory manager. Every object that can be stored and retrieved through the
memory management system inherits from ``MAbstractDataItem``.

Items are created in ``produceData()`` and then handed off to the memory manager,
which takes ownership and caches them for future reuse.


The MAbstractDataItem interface
--------------------------------

``MAbstractDataItem`` is the base class every data item must inherit from.
It provides the bookkeeping that the memory manager needs, plus one pure virtual
method that every subclass must implement::

    virtual size_t getMemorySize_KiB() const = 0;

This method must return the total memory footprint of the item in kibibytes,
including any dynamically allocated sub-structures (arrays, nested objects,
etc.). The memory manager uses this value to enforce the memory budget and
decide which items to evict.

Two further methods track how and where the item was stored. Both are set
automatically by the memory manager and do not need to be called manually:

* ``getGeneratingRequest()`` / ``setGeneratingRequest(MDataRequest)`` records
  the request that produced this item.
* ``getStoringObject()`` / ``setStoringObject(MAbstractDataSource*)`` records
  the data source that owns the item.

``MAbstractDataItem`` itself inherits from ``MMemoryManagementUsingObject``,
which is the common base for both data items and data sources. It assigns each
instance a unique integer ID via ``getID()``, which is used internally by the
memory manager.


Items cannot be shared across sources
-------------------------------------

The memory manager keys every cached item by the pair ``(source, request)``.
A data item is owned by exactly one data source; it cannot be registered in the
memory manager under a different source, and it must not be returned from
``produceData()`` of a source that did not originally create it.

If two sources need access to the same underlying data, the downstream source
should request it from the upstream source, and receive a data lease (see below),
rather than sharing the raw item pointer.


MWeatherPredictionMetaData
---------------------------

For NWP data, items typically also inherit from ``MWeatherPredictionMetaData``.
This mixin adds the standard NWP metadata fields:

* ``initTime`` and ``validTime`` (``QDateTime``)
* ``variableName`` (``QString``)
* ``ensembleMember`` (``int``)

They are set via ``setMetaData(initTime, validTime, variableName,
ensembleMember)`` and read back through the corresponding getters.

It also carries a *processing information* record, a list of key/value
pairs that document what operations were applied to produce the item. These 
need to be set by the producing source. This information is written as 
attributes to an exported NetCDF file.

Data items referenced inside other items
-----------------------------------------

Some data items hold a reference to another data item as a member. A typical
example is ``MLonLatHybridSigmaPressureGrid``, which contains a
``MDataLease<MRegularLonLatGrid> surfacePressure`` member for the surface
pressure field needed during pressure-level computation. Similarly,
``MLonLatAuxiliaryPressureGrid`` holds a
``MDataLease<MLonLatAuxiliaryPressureGrid> auxPressureField_hPa`` member.

``MDataLease<T>`` is a RAII handle that *pins* a cached item so the memory
manager cannot evict it. As long as the lease exists, the referenced item stays
in memory. When the outer item (e.g., the hybrid-sigma grid) is itself evicted
and destroyed, its ``MDataLease`` member is destroyed too, releasing the pin on
the inner item. See :doc:`/05_developer_manual/01_system_modules/memory_management`
for more details.

This has two practical consequences when implementing such a data item:

1. The inner item must be acquired as a lease (not a raw pointer) before it is
   stored in the outer item.
2. ``getMemorySize_KiB()`` of the outer item must *not* count the memory of the
   inner item a second time; the inner item is already tracked independently by
   the memory manager.