Clusters

Open In Colab

A cluster is the most basic form of compute in Runhouse, largely representing a group of instances or VMs connected with Ray. They largely fall in two categories:

  1. Static Clusters: Any machine you have SSH access to, set up with IP addresses and SSH credentials.

  2. On-Demand Clusters: Any cloud instance spun up automatically for you with your cloud credentials.

Runhouse provides various APIs for interacting with remote clusters, such as terminating an on-demand cloud cluster or running remote CLI or Python commands from your local dev environment.

Let’s start with a simple example using AWS. First, install runhouse with AWS dependencies:

! pip install "runhouse[aws]"

Make sure your AWS credentials are set up:

! aws configure
! sky check

On-Demand Clusters

We can start by using the rh.cluster factory function to create our cluster. By specifying an instance_type, Runhouse sets up an On-Demand Cluster in AWS EC2 for us.

Each cluster must be provided with a unique name identifier during construction. This name parameter is used for saving down or loading previous saved clusters, and also used for various CLI commands for the cluster.

Our instance_type here is defined as CPU:2, which is the accelerator type and count that we need (another example would be A10G:2). We could alternatively specify a specific specific instance type, such as p3.2xlarge or g4dn.xlarge (these are instance types on AWS).

import runhouse as rh

aws_cluster = rh.cluster(name="test-cluster", instance_type="CPU:2")

Next, we set up a basic function to throw up on our cluster. For more information about Functions & Modules that you can put up on a cluster, see Functions & Modules.

def run_home(name: str):
    return f"Run home {name}!"

remote_function = rh.function(run_home).to(aws_cluster)

After running .to, your function is set up on the cluster to be called from anywhere. When you call remote_function, it executes remotely on your AWS instance.

remote_function("in cluster!")
INFO | 2024-03-06 15:18:58.439252 | Calling run_home.call
INFO | 2024-03-06 15:18:59.490122 | Time to call run_home.call: 1.05 seconds
'Run home in cluster!!'

On-Demand Clusters with TLS exposed

In the previous example, the cluster that was brought up in EC2 is only accessible to the original user that has SSH credentials to the machine. However, you can set up a cluster with ports exposed to open Internet, and access objects and functions via curl.

tls_cluster = rh.cluster(name="tls-cluster",
                         instance_type="CPU:2",
                         open_ports=[443], # expose HTTPS port to public
                         server_connection_type="tls", # specify how runhouse communicates with this cluster
                         den_auth=False, # no authentication required to hit this cluster (NOT recommended)
)
WARNING | 2024-03-06 15:19:05.297411 | /Users/rohinbhasin/work/runhouse/runhouse/resources/hardware/on_demand_cluster.py:317: UserWarning: Server is insecure and must be inside a VPC or have den_auth enabled to secure it.
  warnings.warn(
remote_tls_function = rh.function(run_home).to(tls_cluster)
remote_tls_function("Marvin")
INFO | 2024-03-06 15:26:05.482586 | Calling run_home.call
INFO | 2024-03-06 15:26:06.550625 | Time to call run_home.call: 1.07 seconds
'Run home Marvin!'
tls_cluster.address
'54.172.178.196'
! curl "https://54.172.178.196/run_home/call?name=Marvin" -k
{"data":""Run home Marvin!"","error":null,"traceback":null,"output_type":"result_serialized","serialization":"json"}

This cluster is exposed to the open Internet, so anyone can hit it. If you do want to share functions and apps publically, it’s recommended you set den_auth=True when setting up your cluster, which requires a user to run runhouse login in order to hit the cluster. We’ll enable it now:

tls_cluster.enable_den_auth()
! curl "https://54.172.178.196/run_home/call?name=Marvin" -k
{"data":null,"error":raise PermissionError(\nPermissionError: No Runhouse token provided. Try running $ runhouse login or visiting https://run.house/login to retrieve a token. If calling via HTTP, please provide a valid token in the Authorization header.\n"","output_type":"exception","serialization":null}

If we send our Runhouse Den token as a header, then the request is valid:

! curl "https://54.172.178.196/run_home/call?name=Marvin" -k -H "Authorization: Bearer <YOUR TOKEN HERE>"
{"data":""Run home Marvin!"","error":null,"traceback":null,"output_type":"result_serialized","serialization":"json"}

Static Clusters

If you have existing machines within a VPC that you want to connect to, you can simply provide the IP addresses and path to SSH credentials to the machine.

cluster = rh.cluster(  # using private key
              name="cpu-cluster-existing",
              ips=['<ip of the cluster>'],
              ssh_creds={'ssh_user': '<user>', 'ssh_private_key':'<path_to_key>'},
          )

Useful Cluster Functions

tls_cluster.run(['pip install numpy && pip freeze | grep numpy'])
Warning: Permanently added '54.172.178.196' (ED25519) to the list of known hosts.
Requirement already satisfied: numpy in /opt/conda/lib/python3.10/site-packages (1.26.4)
numpy==1.26.4
[(0,
  'Requirement already satisfied: numpy in /opt/conda/lib/python3.10/site-packages (1.26.4)nnumpy==1.26.4n',
  "Warning: Permanently added '54.172.178.196' (ED25519) to the list of known hosts.rn")]
tls_cluster.run_python(['import numpy', 'print(numpy.__version__)'])
1.26.4
[(0, '1.26.4n', '')]
Previous
Den Quick Start
Next
Functions and Modules