Task

Tasks are the core building blocks of the decentralized algorithms in Manta. A Task defines the specific computations executed on each node in a Swarm. Using the manta_light API, tasks can interact with the node’s local data, access global parameters, and contribute results back to the shared context of the Swarm.

For more information, refer to the Task API Documentation.

Interations with local data and shared data

The manta_light API provides three main interfaces essential for task development: World, Globals, and Results.

World

World is provided by the Task class. It allows tasks to interact with shared global variables (Globals) and store task-specific outputs (Results). World serves as the communication layer between tasks, enabling synchronization and data sharing across different nodes. For more details, refer to the World API Documentation.

Globals

Globals are global variables shared across all tasks and iterations within the Swarm. These variables can include parameters such as hyperparameters for training or other configuration settings that need to be accessible to all nodes.

There are several rules to know.

Since the variables are shared between tasks in nodes, you can have multiple readers at the same time as shown on the previous figure.

Howewer, if there is one writer or multiple writers at the same time as illustrated on the previous figure, or a writer and multiple readers at the time, you are not guaranteed of an absolute and persistent order of writing and reading on the same value. Therefore, globals are useful when : - multiple readers want to access the same data in global variables - there is only one node writing on a specific variable while other nodes are not accessing its value

For more details, refer to the Globals API Documentation.

Results

Results allow tasks to store outputs that can be accessed in future iterations. For instance, model metrics, trained parameters, or intermediate results can be saved and used by subsequent tasks.

There are several rules to know.

By using results, you can have multiple writes at same time but you add information to the database as many as there are writers.

Reading while writing is possible. Howewer, since the writing and the reading could be unsynchronized, a reading could lead to missing information. For instance, imagine three nodes write their metrics. And a fourth node is starting to write its metrics. But, at the same time, a fifth node selects the metrics, it could receive three metrics from the three first nodes, if the fourth node did not have the time to store its metrics.

For more details, refer to the Results API Documentation.

Local

The Local API is provided by the Task class with access to the node’s local data. These data can include datasets for training, configuration files, or other resources that are stored locally on each node. The Local API abstracts the data storage mechanisms, allowing tasks to focus on processing the data rather than managing data retrieval.

For this version, accessing local data must be specified at first when executing manta_node:

manta_node --data_folder <folder_where_data_are_stored>

For instance :

# In the node

└── data
    └── sensors
        ├── front_images.npz
        └── back_images.npz

manta_node --data_folder data/sensors

Refer to the Local API Documentation for detailed information on interacting with local data.