Shaders in Met.3D

Met.3D uses a fork of GLFX for its shaders. It is a custom GLSL compiler that extends the language with useful features, such as effect files and include files. Effect files combine multiple shader programs into a single file. This makes it easier and more maintainable to write complex shaders, especially if they share common logic, such as NWP variable sampling.

GLFX is no longer being actively developed, which is why we created a fork of it to support Met.3D. The fork mainly supports our updated toolchains using modern CMake, vcpkg, and utilizing Qt6 to resolve OpenGL functions, instead of GLEW.

This page documents the subset of GLFX used throughout Met.3D. It focuses on writing and integrating shaders into the application rather than documenting every GLFX feature.

Throughout this page, standard GLSL knowledge is assumed. Only the GLFX framework and the Met.3D shader framework are described in detail.

Shader files

Shader files in Met.3D are located in the src/glsl/ folder. They can be categorized into two types:

• Effect files (*.fx.glsl) define one or more shader programs. They may also contain uniforms, buffers, constants, structs, and helper functions that are specific to those programs.

• Include files (*.glsl) contain shared shader code but no program definitions. Use them for reusable functions, shared uniform declarations, constants and helper utilities.

• Shared files (*.h) contain constants shared between C++ and shader code. They are valid C/C++ headers and can therefore be included from both languages. They avoid duplicating values such as compile-time constants that must remain consistent on both sides.

General include files that can be used by any effect shader are located inside the src/glsl/includes/ folder. Additionally, shared constants with the c++ code are located inside the src/glsl/shared/ folder.

Writing GLFX shaders

Apart from the extensions described below, GLFX syntax is identical to standard GLSL.

Effect files provide the main entry point into your shader programs. They contain program definitions and can include other files. A minimal effect file can look like this:

Listing 23 A minimal GLFX effect file.

#include "includes/scene.glsl" // Include a file.

// Define vertex shader with one input,
// a 3 component float vector at vertex attribute location 0.
// A single output is also defined, to pass data to the fragment shader.
shader VSmain(in vec3 pos : 0, out float data)
{
    // Implement your vertex shader.
}

shader FSmain(in float data, out vec4 fragColor : 0)
{
    // Implement your fragment shader.
}

// Define the shader program
program ExampleShader
{
    vs(430)=VSmain();
    fs(430)=FSmain();
}

This minimal example should showcase the special GLFX effect file syntax relatively well. See effect_template.fx.glsl for a complete template, or simple_mesh_shader.fx.glsl for a simple shader with multiple programs.

• Include an external file on line 1.

• Define a vertex shader at line 6.

• Define a fragment shader at line 11, which outputs to the first bound color attachment. The suffix : 0 is not strictly required for the parameter to match the color attachment.

• Define the combined shader program at lines 17 to 21.

Includes are resolved relative to the current effect file.

The shader definitions contain your input and output parameters. These parameters define the interface between shader stages. Outputs of one stage must match the corresponding inputs of the next stage.

Interface blocks should be used whenever interpolation qualifiers such as flat or smooth are required, since GLFX does not allow qualifiers directly on shader parameters:

interface VStoFS
{
    smooth float data;
}

VSmain(in vec3 pos : 0, out VStoFS data)
{
    ...
}

FSmain(in VStoFS data, out vec4 fragColor)
{
    ...
}

The program definition specifies the programmable pipeline stages, their GLSL version, and the shader entry point used for each stage. The example program defines two pipeline stages, a vertex shader (vs) with GLSL version 430, which runs the shader VSmain. And a fragment shader defined the same way. It is important to note, that the OpenGL version directive (#version) is no longer necessary with GLFX files, as the program definition contains it.

The supported pipeline stages are:

• vs – Vertex shader

• hs – Hull shader

• ds – Domain shader

• gs – Geometry shader

• fs – Fragment shader

• cs – Compute shader

To declare layout specifiers, such as required by the geometry stage, use the following syntax:

gs(430)=GSmain() : in(triangles_adjacency, invocations=6),
                 out(triangle_strip, max_vertices=18, stream=1);

At runtime, GLFX compiles an effect file into one or more ordinary OpenGL shader programs. Each program block in the effect file corresponds to one OpenGL program object. MShaderEffect manages these compiled programs and provides convenience functions for selecting programs and updating uniforms.

The features described above cover all GLFX functionality currently used by Met.3D. GLFX does have some additional unused features, but these are documented inside the GLFX repository.

Include files

GLFX only resolves includes listed directly in an effect file. Include files therefore do not automatically include their own dependencies, and all required headers must be listed explicitly.

For example, the file oit.glsl also requires scene.glsl and random.glsl, which are listed at the top of the file. These should be included before oit.glsl.

Met.3D shader framework

Met.3D provides an existing shader framework that provides shared uniforms, NWP variable sampling, or other helpers.

Scene uniforms

When rendering a scene in a view, the view already binds all view related data via a uniform buffer. It can be accessed via includes/scene.glsl:

#include "includes/scene.glsl"

shader VSmain(in vec4 pos)
{
    // Access the scene's camera matrix and transform the world position to clip space.
    gl_Position = scene.mvpMatrix * pos;
}

The scene uniform block (SceneData) is always bound at binding point 0. It contains the following notable members:

Member	Description
mvpMatrix	Model-view-projection matrix
cameraPositionWS	View camera position (World space)
cameraForwardWS	View camera direction (World space)
renderCameraPositionWS	Current render camera position (World space)
renderCameraForwardWS	Current render camera direction (World space)
pToWorldZParams	Parameters to convert world Z to pressure and vice-versa.

The file also contains helper functions to convert between world Z and pressure.

Order-independent transparency

To utilize order-independent transparency in your shader, you need to include includes/oit.glsl and its requirements. Afterwards, you can insert your fragment into the OIT algorithm as follows:

#include "includes/scene.glsl"
#include "includes/random.glsl"
#include "includes/oit.glsl"

...

FSmain(out vec4 fragColor)
{
    fragColor = vec4(1.0f, 1.0f, 1.0f, 0.5f);

    // apply oit
    oit(fragColor, gl_FragCoord, gl_SampleMask[0]);
}

The helper function oit() takes the color to insert, the fragment coordinate and the sample mask as inputs. The helper internally selects the configured OIT implementation, allowing algorithms to be changed without modifying shader code.

Shaders should not implement an OIT algorithm directly. Always use oit() so the active transparency implementation can be selected at runtime.

Lighting

The file includes/lighting.glsl provides the lighting model used in our scenes. It defines the light structure, an SSBO containing active scene lights, and helper functions for evaluating scene lighting and shadows.

It depends on includes/scene.glsl. Include the file whenever a shader performs scene lighting:

#include "includes/scene.glsl"
#include "includes/lighting.glsl"

The lighting implementation is shared between shaders. New lighting models or changes to existing ones should therefore be implemented in lighting.glsl rather than duplicated across effect files.

lighting.glsl provides several helper functions for different lighting use cases:

• getBlinnPhongColor() – Calculates the lighting of the current fragment via our Blinn-Phong implementation. It uses all scene lights and applies the shadow maps of the view. Multiple overloads are provided, with which you can control the result, such as optional specular parameters.

• getSimpleBlinnPhongColor() – The same as the normal Blinn-Phong implementation, except it ignores scene lights and shadow maps. It is intended to be used for gizmos, which are not part of the scene, but should still be shaded for spatial perception.

• getSimpleShading() – Applies the contribution of all scene lights while ignoring the surface normal and ambient term. Shadow maps are still applied.

• getShadow() – Only calculates the shadow at the current fragment. The returned shadow factor can be multiplied with the fragment color.

All helpers that apply shadow maps will also apply volumetric shadows from a direct-volume raycaster if present in the scene. If this is not desired, define NO_VOLUMETRIC_SHADOW_SAMPLING before including lighting.glsl.

lighting.glsl also exposes the active scene lights through the shader storage buffer sceneLights. The provided helper functions evaluate lighting using the active lights stored in sceneLights. Most shaders should use the provided helper functions instead of accessing the SSBO directly. Direct access is primarily intended for custom lighting implementations.

An example of using the Blinn-Phong lighting in your shader is the following:

#include "includes/scene.glsl"
#include "includes/lighting.glsl"

...

FSmain(in vec3 worldPos, in vec3 normal, out vec4 fragColor)
{
    fragColor = vec4(1.0f, 1.0f, 1.0f, 0.5f);

    // Evaluate scene lighting.
    fragColor = getBlinnPhongColor(worldPos, normal, fragColor);
}

Random number generation and hashing

The file includes/random.glsl provides some common hashing algorithms. If you require more than the ones defined, add them to this file instead of declaring them inside your effect files.

NWP variable sampling

There is currently no unified abstraction for sampling NWP data. Existing shaders instead share helper includes such as the various volume_sample_*.glsl includes.

Using Shaders

Shaders are wrapped in our GL::MShaderEffect abstraction. See also Shader effects.

To use your shaders you first need to generate an effect program. This is done by the OpenGL Resource management.

Listing 24 Generating an effect program.

// Access GL resources manager
auto *glRM = MGLResourcesManager::getInstance();

// Create a shared pointer to hold the shader effect.
std::shared_ptr<GL::MShaderEffect> effect;

// Generate the shader effect or re-use the cached one.
bool exists = glRM->generateEffectProgram("some_shareable_id", effect);

This creates a new effect program. The identifier acts as a cache key. Actors that use the same effect file should use the same identifier so that the compiled shader can be shared. If a program with the same ID already exists, you can re-use the existing effect and don’t need to compile your shader.

Otherwise, to compile your shader, which is located in the src/glsl/ directory:

Listing 25 Compiling a shader effect file.

//...

bool exists = glRM->generateEffectProgram("some_shareable_id", effect);

if (!exists)
{
    // Compile the shader effect.
    effect->compileFromFile_Met3DHome("src/glsl/your_effect.fx.glsl");
}

If compilation fails, GLFX prints compile errors to the application log. Syntax errors detected by GLFX preserve the original source file and line numbers. Errors reported by the GLSL compiler instead use the format 0:25, where the first number identifies the source file in include order and the second is the line number within that file.

An effect file may define multiple shader programs. Before rendering, bind the program you wish to use:

effect->bindProgram("ProgramName");

The shader effect class also contains helpers to bind various types as named uniforms:

effect->setUniformValue("name", value);

Reloading shaders

You can reload shaders at runtime from disk for all actors present in a scene view a hot-key (Default: L). Individual actors can reload their shaders from the property tree (Development). It calls MActor::reloadShaderEffects() and MActorComponent::reloadShaderEffects() for all attached components.

For view shaders, the view’s properties (Rendering) allow to reload them. This calls MSceneViewGLWidget::reloadShaders().

Re-compile your shaders from source in these functions to be able to hot-reload them.