Data requests

A data request identifies exactly one data item to be produced by a data source. It encodes all parameters that the source (and its upstream sources) need to determine what to compute: the variable name, level type, time steps, ensemble member, and any processing parameters added by filters in the pipeline.

Source code: src/data/datarequest.h, src/data/datarequest.cpp.

Encoding format

MDataRequest is a plain typedef QString. Requests are serialised as a semicolon-separated list of KEY=value pairs, sorted alphabetically by key:

INIT_TIME=2024-01-01T00:00:00;LEVELTYPE=2;MEMBER=0;VALID_TIME=2024-01-01T12:00:00;VARIABLE=air_temperature;

The sorted order is important: two requests that encode the same key/value pairs always produce the same string, so the string can be used directly as a cache key (together with the data source pointer that produced it).

Requests are never constructed or parsed by hand. Use MDataRequestHelper instead.

MDataRequestHelper

MDataRequestHelper is the builder and parser for requests. Construct one from an existing request string to read its keys, build one from scratch to assemble a downstream request, or modify an in-flight request inside produceData().

Constructing and serialising

// Empty request
MDataRequestHelper rh;

// Parse existing request
MDataRequestHelper rh(request);

// Serialise back to MDataRequest
MDataRequest req = rh.request();

Inserting values

insert() accepts the most common Qt and C++ value types:

rh.insert("VARIABLE",    QString("wind_speed"));
rh.insert("LEVELTYPE",   PRESSURE_LEVELS_3D);   // enum → int string
rh.insert("MEMBER",      0);                    // int
rh.insert("SMOOTH_STD",  12.5f);                // float
rh.insert("VALID_TIME",  QDateTime(...));        // ISO 8601
rh.insert("BBOX",        QVector3D(10, 20, 30)); // "x/y/z"

Property objects (MFloatProperty etc.) can also be passed directly as they are implicitly cast to the underlying type.

Reading values

QString  var  = rh.value("VARIABLE");
int      lt   = rh.intValue("LEVELTYPE");
float    std  = rh.floatValue("SMOOTH_STD");
QDateTime vt  = rh.timeValue("VALID_TIME");

Removing keys

A processing source strips its own keys from the request before forwarding it upstream, so the upstream source only sees the keys it declared:

// Remove all keys this source consumes
rh.removeAll(locallyRequiredKeys());

// Keep only the keys relevant to an upstream source
rh.removeAllKeysExcept(upstreamSource->requiredKeys());

Merging requests

unite() merges all key/value pairs from another helper into this one:

rh.unite(otherHelper);

Required keys

Every data source declares which keys must be present in a request addressed to it. The full set of required keys is built automatically from two contributions:

locallyRequiredKeys(): the keys this source itself consumes (e.g. {"SMOOTH"} for a smoothing filter).
The requiredKeys() of every registered input source, optionally prefixed (see below).

The combined list is returned by requiredKeys() and is used to strip irrelevant keys before the request is used as a cache key.

Prefixed requests

When a data source has more than one input, or when it needs to address the same upstream source twice with different parameters, it registers each input with a unique prefix:

registerInputSource(sourceA, "A_");
registerInputSource(sourceB, "B_");

The framework then automatically prepends that prefix to all of sourceA’s required keys when building the combined requiredKeys() list. A caller must therefore include prefixed keys in the request:

A_VARIABLE=u;A_LEVELTYPE=2;A_INIT_TIME=...;B_VARIABLE=v;B_LEVELTYPE=2;B_INIT_TIME=...;

Inside produceData(), use subRequest() to extract the sub-request for each input by stripping the prefix:

MDataRequestHelper rh(request);

// Extract the sub-request for sourceA (strips the "A_" prefix from all keys)
MDataRequest reqA = rh.subRequest("A_").request();
MDataLease<MStructuredGrid> gridA = sourceA->getData(reqA);

MDataRequest reqB = rh.subRequest("B_").request();
MDataLease<MStructuredGrid> gridB = sourceB->getData(reqB);

addKeyPrefix() / removeKeyPrefix() operate on all keys in a helper in place, which is useful when constructing the combined request on the calling side.