Mapper

A Mapper is a built-in Module for mapping a function or module method over a list of inputs, across replicas.

Mapper Factory Method

runhouse.mapper(module: Module, method: str | None = None, num_replicas: int = -1, replicas: List[Module] | None = None, sync_workdir: bool = False, **kwargs) Mapper[source]

A factory method for creating Mapper modules. A mapper is a module that can map a function or module method over a list of inputs in various ways.

Parameters:
  • module (Module) – The module or function to be mapped.

  • method (Optional[str], optional) – The method of the module to be called. If the module is already a callable, this value defaults to "call".

  • num_replicas (int, optional) –

    The number of replicas to run the map function across. (Default: -1)

    • If num_replicas is -1, it will be set to the number of available CPUs (according to Ray).

    • If num_replicas is greater than the number of user-specified replicas (replicas), the remaining replicas will be auto-generated by duplicating the module.

    • If num_replicas is 0, it will be left as the number of user-replicas passed into the module.

    • If num_replicas is less than the number of user-specified replicas, only num_replicas will be used.

  • replicas (Optional[List[Module]], optional) – List of user-specified replicas.

  • sync_workdir (bool, optional) – Whether to sync the working dir to the replicated environments. (Default: False)

Returns:

The resulting Mapper object.

Return type:

Mapper

Example

Mapper Class

class runhouse.Mapper(module: Module, method: str, num_replicas: int | None = -1, replicas: List[Module] | None = None, **kwargs)[source]
__init__(module: Module, method: str, num_replicas: int | None = -1, replicas: List[Module] | None = None, **kwargs)[source]

Runhouse Mapper object. It is used for mapping a function or module method over a list of inputs, across a series of replicas.

Note

To create a Mapper, please use the factory method mapper().

call(*args, **kwargs)[source]

Call the function or method on a single replica.

Example

map(*args, **kwargs)[source]

Map the function or method over a list of arguments.

Example

starmap(args_lists: List, **kwargs)[source]

Like map() except that the elements of the iterable are expected to be iterables that are unpacked as arguments. An iterable of [(1,2), (3, 4)] results in func(1,2), func(3,4)].

Example