Manta Cluster Connection and Swarm Deployment ============================================= Connecting to a Manta Cluster ----------------------------- The :code:`Cluster` object is used to control swarms, manage nodes, and access results and logs within the Manta platform. Below is an explanation of how to connect to a Manta cluster, both in unsecure and secure modes, using the :code:`connect` method. **Cluster Object API Documentation** For more information about the :code:`Cluster` object, refer to the :class:`Cluster ` API Documentation. **Connecting to a Cluster** To connect to a Manta cluster, you use the :code:`Cluster.connect()` method, which creates a gRPC client for communication with the cluster manager. Unsecure Connection ******************* An unsecure connection can be established using the default host and ports without specifying a security token. This mode is suitable for development or testing in a controlled environment. **Example: Unsecure Connection** .. code-block:: python from manta import Cluster import asyncio # Connect to the cluster using default host and ports cluster = asyncio.run(Cluster.connect(host="localhost", port=50051)) # Check if the cluster is available response = asyncio.run(cluster.is_available()) print(response.message) # Output should indicate availability Secure Connection ***************** A secure connection requires a :code:`secured_token`, which is used for authentication when connecting to the cluster manager. This mode is recommended for production environments where security is a concern. **Example: Secure Connection** .. code-block:: python from manta import Cluster import asyncio # Connect securely using a secured token secured_token = "YOUR_SECURED_TOKEN" cluster = asyncio.run( Cluster.connect( host="localhost", port=50051, ca_port=50050, secured_token=secured_token ) ) # Check if the cluster is available response = asyncio.run(cluster.is_available()) print(response.message) # Output should indicate availability .. warning:: Once you have been connected successfully to your cluster, :code:`secured_token` must not be filled in. Instead, local certificated files will be used to access securely your cluster. Deploying Swarms Using the Cluster Object ----------------------------------------- Once you have successfully connected to the cluster, you can deploy swarms for execution on the nodes. The deployment process involves sending the swarm to the manager, registering it, and starting it. Below is a step-by-step guide to deploying a swarm using the :code:`Cluster` object. 1. **Initialize the Cluster**: Connect to the Manta Manager by initializing the :code:`Cluster` object with the appropriate host and port. 2. **Check Cluster Availability**: Ensure that the cluster is available using the :code:`is_available()` method. 3. **Send the Swarm**: Use the :code:`send_swarm()` method to register your swarm with the manager. 4. **Start the Swarm**: Use the :code:`start_swarm()` method to initiate the swarm's execution. **Example: Swarm Deployment and Execution** .. code-block:: python import asyncio from manta import Cluster from swarm import FLSwarm async def deploy_swarm(): # Initialize the Cluster object with the Manager's host and port cluster = await Cluster.connect(host="localhost", port=50051, ca_port=50050) # Ensure the cluster is available response = await cluster.is_available() print(response.message) # Should print "Available" # Define a Swarm object, e.g., a Federated Learning Swarm swarm = FLSwarm(n_workers=10) # Send the Swarm to the Manager swarm_response = await cluster.send_swarm(swarm) swarm_id = swarm_response.swarm_id print(f"Swarm registered with ID: {swarm_id}") # Allow some time for the Swarm to be registered await asyncio.sleep(1) # Start the Swarm start_response = await cluster.start_swarm(swarm_id) print(start_response.message) # Should confirm that the swarm has started # Run the deployment process asyncio.run(deploy_swarm()) **Explanation of Key Methods** - **Cluster Initialization**: Connects to the Manta Manager at the specified :code:`host` and :code:`port`. - **Swarm Deployment**: The :code:`send_swarm()` method registers the swarm with the manager. The :code:`swarm_id` returned can be used to track the swarm's status. - **Swarm Execution**: The :code:`start_swarm()` method triggers the execution of the swarm on the connected nodes. The response indicates whether the swarm has started successfully. Accessing Swarm Results ----------------------- Once the swarm is deployed and running, you can retrieve results and logs using the :code:`Cluster` object. This allows for monitoring and managing the swarms effectively. **Example: Accessing Swarm Logs and Results** .. code-block:: python # Example code to access logs or results after swarm execution logs = await cluster.collect_logs(swarm_id) print(logs) # Output swarm logs .. note:: See :meth:`collect_logs ` for more information. For detailed API calls and additional options, refer to the :class:`Cluster ` API Documentation. With this guide, you should be able to connect to a Manta cluster, deploy swarms, and monitor their execution effectively.