Scheduling in Met.3D
====================

The Met.3D scheduling system is responsible for efficiently managing computational tasks. 
It ensures that tasks are executed in the correct order while handling dependencies 
between them. The scheduling mechanism is designed to support **multi-threading**, 
**GPU tasks**, and **resource-aware execution** to optimize performance.

Overview
--------

Tasks in Met.3D are represented as nodes in a **task graph**. Each task can depend on 
one or more parent tasks and may have multiple child tasks. The scheduler determines 
the execution order based on dependencies and system resource constraints.

Key Concepts
~~~~~~~~~~~~

- **Task Graph**: Represents dependencies between tasks in a **Directed Acyclic Graph (DAG)**.  
  Nodes without parent dependencies can be executed as soon as they meet the scheduler’s criteria.  
  Once a task completes, it is removed from the task graph.
- **State Management**: Tasks transition through multiple states (``PENDING``, ``SUBMITTED``, ``SCHEDULED``,  
  ``IN_PROGRESS``, ``COMPLETED``), ensuring proper execution flow.
- **Multi-threaded Scheduler**: Manages scheduling and execution of tasks across multiple threads.
- **Task Graph Handler**: Runs in a separate thread, controlling access to the task graph and managing task execution safely.

Task Management: ``MTask``
--------------------------

The core scheduling unit is the ``MTask`` class, which represents a single computational task.  
It defines dependencies between tasks, execution flow, and resource requirements.

Key Features of ``MTask``
~~~~~~~~~~~~~~~~~~~~~~~~~

- **Task States**: Tasks transition through multiple states (``PENDING``, ``SUBMITTED``, ``SCHEDULED``, 
  ``IN_PROGRESS``, ``COMPLETED``).
- **Dependency Management**: Each task may have parent and child tasks, forming a **task graph**.
- **Execution Control**: Tasks execute via `run()`, which invokes `MScheduledDataSource::processRequest()`.
- **Resource Awareness**: Tasks can be flagged as **GPU** or **disk reader** tasks for optimized execution.
- **Progress Tracking**: Tasks can be linked to a **progress bar** to provide user feedback.

Task Lifecycle
~~~~~~~~~~~~~~

1. **Creation**: A new `MTask` instance is created with an ``MDataRequest`` and a data source.
2. **Submission**: The task is submitted to the scheduler and marked as ``SUBMITTED``.
3. **Scheduling**: When all dependencies are resolved, the task is marked as ``SCHEDULED``. When a task
   is being scheduled, all parent tasks are already either scheduled, running, or completed.
4. **Execution**: The task performs its computation or data retrieval (``IN_PROGRESS``).
5. **Completion**: Once execution finishes, the task is marked as ``COMPLETED``.

Task Dependencies and Graph Structure
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

- The **task graph** organizes tasks hierarchically, ensuring dependencies are met before execution.
- The **Task Graph Handler** ensures **thread-safe** access to task dependencies via mutex locks.
- **Dynamic dependency management** allows tasks to adjust their dependencies and memory reservations as needed.

Scheduler and Execution Flow
----------------------------

The scheduler manages task execution by resolving dependencies, allocating resources, and executing tasks efficiently.

Task Graph Handler (``MTaskGraphHandler``)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The **Task Graph Handler** runs in a **dedicated thread**. It manages task scheduling and execution, ensuring:

- **Thread safety**: The task graph is only accessed through this handler, making it the **sole** controller.
- **Task readiness checking**: Uses ``acceptTaskToRun()`` to determine if a task is executable.
- **Task execution**: Once dependencies are resolved, it informs the scheduler to execute the task.

GPU and Threaded Execution
~~~~~~~~~~~~~~~~~~~~~~~~~~

- **GPU tasks** always execute in the **main thread**.
- **All other tasks** are executed in **separate threads**, which are destroyed after execution.

The Met.3D scheduler: ``MScheduler``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

- The ``MScheduler`` is the singleton scheduler in Met.3D.
- Dynamically schedules and executes tasks based on available system resources.
- It can be configured using the ``--threads`` command-line option:
  
  - ``--threads <N>``: Limits execution to **N** concurrent spawned threads.
  - ``--threads 0``: Runs **all** tasks in the **main UI thread**.

.. toctree::
    :maxdepth: 1
    :hidden: