A Folder represents a specified location for organizing and storing other Runhouse primitives across various systems.
Creates a Runhouse folder object, which can be used to interact with the folder at the given path.
name (Optional[str]) – Name to give the folder, to be re-used later on.
path (Optional[str or Path]) – Path (or path) that the folder is located at.
system (Optional[str or Cluster]) – File system or cluster name. If providing a file system this must be one of:
[file, github, sftp, ssh, s3, gs, azure].
We are working to add additional file system support.
dryrun (bool) – Whether to create the Folder if it doesn’t exist, or load a Folder object as a dryrun.
(Default: False)
local_mount (bool) – Whether or not to mount the folder locally. (Default: False)
data_config (Optional[Dict]) – The data config to pass to the underlying fsspec handler.
The resulting folder.
Example
>>> import runhouse as rh
>>> rh.folder(name='training_imgs', path='remote_directory/images', system='s3').save()
>>> # Load folder from above
>>> reloaded_folder = rh.folder(name="training_imgs")
Whether path of a Folder exists locally.
Example
>>> my_folder = rh.folder("local/folder/path")
>>> in_folder = my_folder.contains("filename")
Returns a new Folder object pointing to the destination folder.
Whether the folder exists in the filesystem.
Example
>>> exists_on_system = my_folder.exists_in_system()
Generate the FSSpec URL using the file system and path of the folder
Returns the contents of a file as a string or bytes.
Example
>>> contents = my_folder.get(file_name)
Whether the folder is on the local filesystem.
Example
>>> is_local = my_folder.is_local()
Whether the folder is writable.
Example
>>> if my_folder.is_writable():
>>> ....
Locate the local path of a Folder given an rns path.
Example
>>> my_folder = rh.folder("local/folder/path")
>>> local_path = my_folder.locate("file_name")
List the contents of the folder.
full_paths (Optional[bool]) – Whether to list the full paths of the folder contents.
Defaults to True.
sort (Optional[bool]) – Whether to sort the folder contents by time modified.
Defaults to False.
Create the folder in specified file system if it doesn’t already exist.
Mount the folder locally.
Example
remote_folder = rh.folder(“folder/path”, system=”s3”) local_mount = remote_folder.mount()
Move the folder to a new filesystem or cluster.
Example
>>> folder = rh.folder(path="local/path")
>>> folder.mv(my_cluster)
>>> folder.mv("s3", "s3_bucket/path")
Returns an fsspec file, which must be used as a content manager to be opened.
Example
>>> with my_folder.open('obj_name') as my_file:
>>> pickle.load(my_file)
Put given contents in folder.
contents (Dict[str, Any] or Resource or List[Resource]) – Contents to put in folder. Must be a dict with keys being the file names (without full paths) and values being the file-like objects to write, or a Resource object, or a list of Resources.
overwrite (bool) – Whether to dump the file contents as json. By default expects data to be encoded.
Defaults to False.
mode (Optional(str)) – Write mode to use for fsspec. Defaults to wb.
write_fn (Optional(Callable)) – Function to use for writing file contents. Example: ``write_fn = lambda f, data: json.dump(data, f)
Example
>>> my_folder.put(contents={"filename.txt": data})
List the resources in the RNS folder.
Example
>>> resources = my_folder.resources()
Delete a folder from the file system. Optionally provide a list of folder contents to delete.
contents (Optional[List]) – Specific contents to delete in the folder.
recursive (bool) – Delete the folder itself (including all its contents).
Defaults to True.
Example
>>> my_folder.rm()
Traverse up the filesystem until reaching one of the directories in rns_base_folders, then compute the relative path to that.
Rsync local folder to remote.