Interaction handles

Interaction handles represent a possible user interaction on an actor or actor component using the mouse cursor. They are required if you want to interact with your actor or component in a view’s interaction mode.

A view uses interaction handles to track what the user is currently interacting with. They can be thought of as interaction descriptors, they are not direct objects in the system, but rather a collection of information. They carry metadata, such as cursor shapes or tooltips, that give the user context for the interaction. Additionally, they also contain functors that execute the interaction, such as dragging and releasing. A handle therefore contains both the interaction behavior and the information required by the view to present that interaction to the user. Through this, it decouples the view’s interaction logic from the actor or component implementing the interaction. The view only needs to select and execute a handle and therefore does not need to know which actor or component provided it.

To guard against use-after-free, each handle keeps a reference to its owner. If the owner is deleted during an interaction, the handle ignores all invocations of its functors.

During interaction mode, a view requests all actors and components to intersect their interaction handles with the mouse cursor. These can return an interaction result that contains the closest interaction handle and the distance to the camera.

The view then selects the closest interaction handle for user interaction.

Fig. 47 The interaction flow

This page details the implementation of the interaction handles and how to use them.

Implementation

Interaction handles and other interaction related structs are implemented in gxfw/minteractionhandle.h.

Handles

MInteractionHandle describes a single possible mouse interaction. It currently supports mouse drags and hovering.

The handle provides several functors that are called at different stages of an interaction:

• MInteractionHandle::dragEvent(const MInteractionContext&, DragEventID) – Called when this handle is being dragged by the user. This is called on every event loop update while the user is dragging the mouse.

• MInteractionHandle::hoverEvent() – (Optional) Called when this handle is the closest handle to the camera and therefore the one hovered by the mouse. It is called on every event loop update while the handle is considered hovered and the primary mouse button is not pressed.

• MInteractionHandle::releaseEvent(DragEventID) – (Optional) Called when this handle is being released by the user. This is called once the user releases the primary mouse button, if this handle was the one being dragged.

• MInteractionHandle::getPositionLabel(const MInteractionContext&) – (Optional) Called to get an up-to-date position label of this handle. The view gains ownership of the returned label.

Note

MInteractionHandle::getPositionLabel returns an MLabelScoped. These are created by calling MTextManager::makeScoped(MLabel*).

The optional functors can be left unset if the functionality is not required for your handle. These functors are never executed directly. Instead, the handle provides an invoke method for each functor that will safely call the functor. These always check if the owner has not been destroyed before executing the functor, as the functor might reference the deleted owner.

Additionally, a handle contains the following metadata fields:

• MInteractionHandle::owner – (Required) The owner of this handle. This is a QPointer to the owner, which is automatically cleared when the owner is destroyed. Once the owner is destroyed, the handle’s interactions are ignored. This is required, as a handle only describes an interaction, but that interaction might not be possible if the owner was deleted. If your callbacks capture this, e.g. [this](){}, then a use-after-free could happen calling the callback after this was deleted. The QPointer is therefore required even when a callback captures this, as the capture itself does not prevent the callback from being invoked after the owner has been destroyed.

• MInteractionHandle::tooltip – (Optional) A tooltip to show when the handle is hovered.

• MInteractionHandle::hoverCursor – (Optional) A cursor shape to use when the handle is hovered.

• MInteractionHandle::dragCursor – (Optional) A cursor shape to use when the handle is being dragged.

Context

Interactions are always given an MInteractionContext. This struct contains information relevant for an interaction, such as the view that was interacted with and the mouse cursor location. It is passed to MInteractionHandle::dragEvent and MInteractionHandle::getPositionLabel. Using a shared context object instead of passing individual arguments to each functor also provides an extension point for the interaction framework. New context information can be added without changing every callback signature.

Currently, it provides the following information:

• MInteractionContext::view – The view in which the interaction takes place.

• MInteractionContext::mouseClipPos – The position of the mouse in clip-space. This can be used to project the mouse position into world space for intersection checks.

• MInteractionContext::mouseViewportPos – The position of the mouse in pixel coordinates.

Intersection results

Actors that perform an intersection check can return an MIntersectionResult. The check is performed by MActor::checkIntersectionWithHandle(const MInteractionContext &context), and MActorComponent::checkIntersectionWithHandle(const MInteractionContext &context) and must be overridden in derived classes. Note, MActor::isPickable() has to return true for interaction to be checked. It can be enabled with MActor::enablePicking(true).

It contains the closest interaction handle and its distance to the camera. The distance is the distance of your handle to the camera in world space. If your handle is used for 2D UI elements, the distance should be 0. Multiple actors and components can provide an interaction handle for the same cursor position. The intersection distance allows the view to compare these results and select the closest handle. It will be the one considered hovered.

Drag events

Since drag events should be undoable, we also provide a DragEventID. It is an 8-bit integer, and therefore provides 256 unique IDs before wrapping around.

The drag event ID is global and remains constant for the duration of a drag. It is incremented when the mouse button is released after a drag.

These are passed to MInteractionHandle::dragEvent and MInteractionHandle::releaseEvent.

Note

The low bit size of DragEventID is deliberate. Since they are intended to identify subsequent Undo command’s, this was deemed sufficient. A collision can only occur if 256 consecutive drag events are created to wrap the 8-bit ID. A drag event in this case is defined between mouse click and release, where the mouse moved. It is not one call to MInteractionHandle::dragEvent. All calls to MInteractionHandle::dragEvent that occour before a MInteractionHandle::releaseEvent will receive the same DragEventID. When creating an undo command from a drag event, pass this ID to allow the undo-system to merge commands belonging to the same drag. See Merging commands for more information.

Usage

To get started using interaction handles, add a handle to your actor or component. It is typically stored as a member of the actor or component that owns the interaction.:

MInteractionHandle moveHandle;

And initialize it in the constructor of your class:

Listing 17 Initialize a basic interaction handle for dragging.

// ...

// Required to set owner, so that the handle cancels interaction if the owner is deleted during interaction.
moveHandle.owner = this;
// Setup callbacks that handle the interaction
moveHandle.dragEvent = [this](const MInteractionContext &context, DragEventID eventID) -> void
{
    // Implement drag logic or call handler of drag logic.
};

// ...

To make your actor or component interactable, you have to implement the intersection check that selects the handle under the mouse cursor. Override the intersection check of your base class and select the closest intersected handle, if any:

Listing 18 Intersection check that selects a handle for interaction.

std::optional<MIntersectionResult> checkIntersectionWithHandle(const MInteractionContext &context) override
{
    // Implement intersection logic using the provided interaction context.
    bool intersectsHandle = ...;
    float intersectionDistance = ...;
    if (intersectsHandle)
    {
        return MIntersectionResult{moveHandle, intersectionDistance};
    }

    // No intersection, return empty optional.
    return {};
}

You can define an optional tooltip and mouse cursor shapes for your handle. These help the user understand the interaction that is possible with the handle:

Listing 19 Customizing handle appearance

// Set a tooltip to tell the user what this handle does, when they hover it.
moveHandle.tooltip = "This handle moves this actor.";
// Set a hover cursor, so that the user knows they are hovering a handle.
moveHandle.hoverCursor = Qt::CursorShape::SizeAllCursor;
// Set a drag cursor that is shown during the drag operation.
moveHandle.dragCursor = Qt::CursorShape::SizeAllCursor;

One important topic is also the creation of a position label for your handle. It requires the label to be offset from the handle to not be obstructed by it. The label offset is calculated relative to the camera so that the label remains visually separated from the handle as the camera distance changes:

Listing 20 Creation of a position label for the handle.

moveHandle.getPositionLabel = [this](const MInteractionContext &context) -> MLabelScoped
{
    // Retrieve text manager.
    MGLResourcesManager *glRM = MGLResourcesManager::getInstance();
    MTextManager *tm = glRM->getTextManager();

    // Get handle world space position.
    QVector3D handlePosWS = ...;

    // Create label
    MLabel * label = tm->addText(...);

    // Project mouse into world.
    QVector3D mousePosWorld = context.view->projectClipSpaceToWorld(context.mouseClipPos,
                                                                    handlePosWS,
                                                                    QVector3D(0, 0, 1));
    // Calculate offset weight, which keeps the label the same distance from the handle in clip-space
    // regardless of the distance of the camera to the handle.
    float weight = MActor::computePositionLabelDistanceWeight(context.view->getCamera(),
                                                              mousePosWorld);

    // Offset label from the handle, in the right direction on screen.
    label->anchorOffset = -(weight * context.view->getCamera()->getXAxis());

    // Return a scoped label that gets automatically removed once no longer needed.
    return tm->makeScoped(label);
};