Swarm

In the Manta platform, a Swarm enables users to develop decentralized and collaborative algorithms distributed across multiple nodes. A Swarm consists of various tasks defined by the user, which are executed collaboratively on nodes. Manta Core offers the tools and abstractions needed to efficiently develop, deploy, and monitor these Swarms, facilitating the seamless execution of complex, distributed workflows.

What is a Swarm ?

Swarm Swarm

A Swarm is a decentralized pipeline composed of interconnected tasks that execute collaboratively across nodes in a distributed system. Each Swarm defines a sequence of operations, organized as tasks, that collectively perform a larger algorithm. The primary feature of a Swarm is its ability to define an iteration that repeats until a specific condition or completion criteria are met. This iterative approach allows for continuous data processing and refinement across multiple stages.

Components of a Swarm

  • Modules: The basic building blocks of a Swarm, each module performs a specific operation or computation within a task. Tasks can depend on each other, allowing complex workflows to be built.

  • Dependencies: Tasks within a Swarm can be interconnected through defined dependencies, creating a directed graph of operations. These dependencies control the flow of data and execution order among tasks.

  • Iteration Cycle: A Swarm is designed to operate in cycles, where the defined tasks are executed repeatedly until the Swarm completes its objective.

How Swarms Work

Swarms work by defining an iterative workflow that is distributed across multiple nodes. In each iteration, tasks process data, communicate with other tasks, and contribute to the overall progress of the Swarm. This setup is particularly useful for collaborative algorithms, where nodes must work together to achieve a shared goal.

The above diagram illustrates a Swarm consisting of seven tasks distributed across nodes. The tasks communicate with each other to pass data and synchronize their operations. After completing one cycle, the Swarm proceeds to the next iteration until it reaches the termination condition.

Swarm Class

To create a new Swarm, you define a Python class that inherits from the manta.swarm.Swarm base class. This custom class specifies the tasks, their dependencies, and the overall structure of the decentralized algorithm.

For more detailed information on implementing a Swarm class, refer to the Swarm API Documentation.