Manta Cluster Connection and Swarm Deployment¶

Connecting to a Manta Cluster¶

The Cluster object is used to control swarms, manage nodes, and access results and logs within the Manta platform. Below is an explanation of how to connect to a Manta cluster, both in unsecure and secure modes, using the connect method.

Cluster Object API Documentation

For more information about the Cluster object, refer to the Cluster API Documentation.

Connecting to a Cluster

To connect to a Manta cluster, you use the Cluster.connect() method, which creates a gRPC client for communication with the cluster manager.

Unsecure Connection¶

An unsecure connection can be established using the default host and ports without specifying a security token. This mode is suitable for development or testing in a controlled environment.

Example: Unsecure Connection

from manta import Cluster
import asyncio

# Connect to the cluster using default host and ports
cluster = asyncio.run(Cluster.connect(host="localhost", port=50051))

# Check if the cluster is available
response = asyncio.run(cluster.is_available())
print(response.message)  # Output should indicate availability

Secure Connection¶

A secure connection requires a secured_token, which is used for authentication when connecting to the cluster manager. This mode is recommended for production environments where security is a concern.

Example: Secure Connection

from manta import Cluster
import asyncio

# Connect securely using a secured token
secured_token = "YOUR_SECURED_TOKEN"
cluster = asyncio.run(
    Cluster.connect(
        host="localhost",
        port=50051,
        ca_port=50050,
        secured_token=secured_token
    )
)

# Check if the cluster is available
response = asyncio.run(cluster.is_available())
print(response.message)  # Output should indicate availability

Warning

Once you have been connected successfully to your cluster, secured_token must not be filled in. Instead, local certificated files will be used to access securely your cluster.

Deploying Swarms Using the Cluster Object¶

Once you have successfully connected to the cluster, you can deploy swarms for execution on the nodes. The deployment process involves sending the swarm to the manager, registering it, and starting it.

Below is a step-by-step guide to deploying a swarm using the Cluster object.

Initialize the Cluster: Connect to the Manta Manager by initializing the Cluster object with the appropriate host and port.
Check Cluster Availability: Ensure that the cluster is available using the is_available() method.
Send the Swarm: Use the send_swarm() method to register your swarm with the manager.
Start the Swarm: Use the start_swarm() method to initiate the swarm’s execution.

Example: Swarm Deployment and Execution

import asyncio
from manta import Cluster
from swarm import FLSwarm

async def deploy_swarm():
    # Initialize the Cluster object with the Manager's host and port
    cluster = await Cluster.connect(host="localhost", port=50051, ca_port=50050)

    # Ensure the cluster is available
    response = await cluster.is_available()
    print(response.message)  # Should print "Available"

    # Define a Swarm object, e.g., a Federated Learning Swarm
    swarm = FLSwarm(n_workers=10)

    # Send the Swarm to the Manager
    swarm_response = await cluster.send_swarm(swarm)
    swarm_id = swarm_response.swarm_id
    print(f"Swarm registered with ID: {swarm_id}")

    # Allow some time for the Swarm to be registered
    await asyncio.sleep(1)

    # Start the Swarm
    start_response = await cluster.start_swarm(swarm_id)
    print(start_response.message)  # Should confirm that the swarm has started

# Run the deployment process
asyncio.run(deploy_swarm())

Explanation of Key Methods

Cluster Initialization: Connects to the Manta Manager at the specified host and port.
Swarm Deployment: The send_swarm() method registers the swarm with the manager. The swarm_id returned can be used to track the swarm’s status.
Swarm Execution: The start_swarm() method triggers the execution of the swarm on the connected nodes. The response indicates whether the swarm has started successfully.

Accessing Swarm Results¶

Once the swarm is deployed and running, you can retrieve results and logs using the Cluster object. This allows for monitoring and managing the swarms effectively.

Example: Accessing Swarm Logs and Results

# Example code to access logs or results after swarm execution
logs = await cluster.collect_logs(swarm_id)
print(logs)  # Output swarm logs

Note

See collect_logs for more information.

For detailed API calls and additional options, refer to the Cluster API Documentation.

With this guide, you should be able to connect to a Manta cluster, deploy swarms, and monitor their execution effectively.