Folder

A Folder represents a specified location for organizing and storing other Runhouse primitives across various systems.

Folder Factory Method

runhouse.folder(name: str | None = None, path: str | Path | None = None, system: str | Cluster | None = None, load_from_den: bool = True, dryrun: bool = False) Folder[source]

Creates a Runhouse folder object, which can be used to interact with the folder at the given path.

Parameters:
  • name (Optional[str]) – Name to give the folder, to be re-used later on.

  • path (Optional[str or Path]) – Path (or path) that the folder is located at.

  • system (Optional[str or Cluster]) – File system or cluster name. If prpre-oviding a file system this must be one of: [file, s3, gs].

  • load_from_den (bool) – Whether to try loading the Folder resource from Den. (Default: True)

  • dryrun (bool) – Whether to create the Folder if it doesn’t exist, or load a Folder object as a dryrun. (Default: False)

Returns:

The resulting folder.

Return type:

Folder

Example

>>> import runhouse as rh >>> rh.folder(name='training_imgs', path='remote_directory/images', system='s3').save()
>>> # Load folder from above >>> reloaded_folder = rh.folder(name="training_imgs")

Folder Class

class runhouse.Folder(name: str | None = None, path: str | None = None, system: str | Cluster | None = None, dryrun: bool = False, **kwargs)[source]
__init__(name: str | None = None, path: str | None = None, system: str | Cluster | None = None, dryrun: bool = False, **kwargs)[source]

Runhouse Folder object.

Note

To build a folder, please use the factory method folder().

contains(name_or_path: str) bool[source]

Whether path exists locally inside a folder.

Parameters:

name_or_path (str) – Name or path of folder to check if it exists inside the folder.

Example

>>> my_folder = rh.folder("local/folder/path") >>> in_folder = my_folder.contains("filename")
exists_in_system()[source]

Whether the folder exists in the filesystem.

Example

>>> exists_on_system = my_folder.exists_in_system()
static from_config(config: Dict, dryrun: bool = False, _resolve_children: bool = True)[source]

Load or construct resource from config.

Parameters:
  • config (Dict) – Resource config.

  • dryrun (bool, optional) – Whether to construct resource or load as dryrun (Default: False)

property fsspec_url

Generate the FSSpec style URL using the file system and path of the folder

get(name, mode: str = 'rb', encoding: str | None = None)[source]

Returns the contents of a file as a string or bytes.

Parameters:
  • name (str) – Name of file to get contents of.

  • mode (str, optional) – Mode for opening the file. (Default: "rb")

  • encoding (str, optional) – Encoding for opening the file. (Default: None)

Example

>>> contents = my_folder.get(file_name)
is_local()[source]

Whether the folder is on the local filesystem.

Example

>>> is_local = my_folder.is_local()
locate(name_or_path) Tuple[str, str][source]

Locate the local path of a Folder given an rns path.

Parameters:

name_or_path (str) – Name or path of folder to locate inside the folder.

Example

>>> my_folder = rh.folder("local/folder/path") >>> local_path = my_folder.locate("file_name")
ls(full_paths: bool = True, sort: bool = False) List[source]

List the contents of the folder.

Parameters:
  • full_paths (Optional[bool]) – Whether to list the full paths of the folder contents. Defaults to True.

  • sort (Optional[bool]) – Whether to sort the folder contents by time modified. Defaults to False.

mkdir()[source]

Create the folder in specified file system if it doesn’t already exist.

mv(system: str | Cluster, path: str | None, overwrite: bool = True) None[source]

Move the folder to a new filesystem or cluster.

Parameters:
  • system (str or Cluster) – Filesystem or cluster to move the folder to.

  • path (str) – Path to move the folder to.

  • overwrite (bool, optional) – Whether to override if a file already exists at the destination path. (Default: True)

Example

>>> folder = rh.folder(path="local/path") >>> folder.mv(my_cluster) >>> folder.mv("s3", "s3_bucket/path")
open(name, mode: str = 'rb', encoding: str | None = None)[source]

Returns the specified file as a stream (botocore.response.StreamingBody), which must be used as a content manager to be opened.

Parameters:
  • name (str) – Name of file inside the folder to open.

  • mode (str, optional) – Mode for opening the file. (Default: "rb")

  • encoding (str, optional) – Encoding for opening the file. (Default: None)

Example

>>> with my_folder.open('obj_name') as my_file: >>> pickle.load(my_file)
put(contents: Dict[str, Any], overwrite: bool = False, mode: str = 'wb')[source]

Put given contents in folder.

Parameters:
  • contents (Dict[str, Any] or Resource or List[Resource]) – Contents to put in folder. Must be a dict with keys being the file names (without full paths) and values being the file-like objects to write, or a Resource object, or a list of Resources.

  • overwrite (bool, optional) – Whether to overwrite the existing file if it exists. (Default: False)

  • mode (str, optional) – Write mode to use. (Default: wb).

Example

>>> my_folder.put(contents={"filename.txt": data})
resources(full_paths: bool = False)[source]

List the resources in the folder.

Parameters:

full_paths (bool, optional) – Whether to list the full path or relative path. (Default: False)

Example

>>> resources = my_folder.resources()
rm(contents: List | None = None, recursive: bool = True)[source]

Delete a folder from the file system. Optionally provide a list of folder contents to delete.

Parameters:
  • contents (Optional[List]) – Specific contents to delete in the folder. If None, removes the entire folder. (Default: None)

  • recursive (bool, optional) – Delete the folder itself (including all its contents). (Default: True)

Example

>>> my_folder.rm()
property rns_address

Traverse up the filesystem until reaching one of the directories in rns_base_folders, then compute the relative path to that.

to(system: str | Cluster, path: str | Path | None = None)[source]

Copy the folder to a new filesystem or cluster. Currently supported: here, file, gs, s3, or a cluster.

Parameters:
  • system (str or Cluster) – Filesystem or cluster to move the folder to.

  • path (str, optional) – Path to move the folder to.

Example

>>> local_folder = rh.folder(path="/my/local/folder") >>> s3_folder = local_folder.to("s3")