Task ==== Tasks are the core building blocks of the decentralized algorithms in Manta. A :code:`Task` defines the specific computations executed on each node in a Swarm. Using the :code:`manta_light` API, tasks can interact with the node’s local data, access global parameters, and contribute results back to the shared context of the Swarm. For more information, refer to the :class:`Task ` API Documentation. .. image:: ../_static/images/light-task.png :alt: Task :align: center :class: only-light .. image:: ../_static/images/dark-task.png :alt: Task :align: center :class: only-dark Interations with local data and shared data ------------------------------------------- The :code:`manta_light` API provides three main interfaces essential for task development: :class:`World `, :class:`Globals `, and :class:`Results `. World ***** :class:`World ` is provided by the :code:`Task` class. It allows tasks to interact with shared global variables (:class:`Globals `) and store task-specific outputs (:class:`Results `). :class:`World ` serves as the communication layer between tasks, enabling synchronization and data sharing across different nodes. For more details, refer to the :class:`World ` API Documentation. Globals ^^^^^^^ :class:`Globals ` are global variables shared across all tasks and iterations within the Swarm. These variables can include parameters such as hyperparameters for training or other configuration settings that need to be accessible to all nodes. There are several rules to know. .. image:: ../_static/images/light-globals-ok.png :align: center :class: only-light .. image:: ../_static/images/dark-globals-ok.png :align: center :class: only-dark Since the variables are shared between tasks in nodes, you can have multiple readers at the same time as shown on the previous figure. .. image:: ../_static/images/light-globals-error.png :align: center :class: only-light .. image:: ../_static/images/dark-globals-error.png :alt: Task :align: center :class: only-dark Howewer, if there is one writer or multiple writers at the same time as illustrated on the previous figure, or a writer and multiple readers at the time, you **are not guaranteed of an absolute and persistent order of writing and reading on the same value**. Therefore, globals are useful when : - multiple readers want to access the same data in global variables - there is only one node writing on a specific variable while other nodes are not accessing its value For more details, refer to the :class:`Globals ` API Documentation. Results ^^^^^^^ :class:`Results ` allow tasks to store outputs that can be accessed in future iterations. For instance, model metrics, trained parameters, or intermediate results can be saved and used by subsequent tasks. There are several rules to know. .. image:: ../_static/images/light-result-both-write.png :align: center :class: only-light .. image:: ../_static/images/dark-result-both-write.png :align: center :class: only-dark By using results, you can have multiple writes at same time but **you add information to the database as many as there are writers**. .. image:: ../_static/images/light-result-read-write.png :align: center :class: only-light .. image:: ../_static/images/dark-result-read-write.png :align: center :class: only-dark Reading while writing is possible. Howewer, since the writing and the reading could be unsynchronized, a reading could lead to missing information. For instance, imagine **three** nodes write their metrics. And a fourth node is starting to write its metrics. But, at the same time, a **fifth** node selects the metrics, it could receive **three** metrics from the three first nodes, if the **fourth** node did not have the time to store its metrics. For more details, refer to the :class:`Results ` API Documentation. Local ***** The Local API is provided by the :code:`Task` class with access to the node's local data. These data can include datasets for training, configuration files, or other resources that are stored locally on each node. The Local API abstracts the data storage mechanisms, allowing tasks to focus on processing the data rather than managing data retrieval. For this version, accessing local data must be specified at first when executing :code:`manta_node`: .. code:: bash manta_node --data_folder For instance : .. code:: # In the node └── data    └── sensors    ├── front_images.npz    └── back_images.npz .. code:: bash manta_node --data_folder data/sensors Refer to the :class:`Local ` API Documentation for detailed information on interacting with local data.