flambe.experiment.utils
¶
Module Contents¶
-
flambe.experiment.utils.
check_links
(blocks: Dict[str, Schema], global_vars: Optional[Dict[str, Any]] = None) → None[source]¶ Check validity of links between blocks.
Ensures dependency order, and that only Comparable blocks are being reduced through a LinkBest object.
Parameters: blocks (OrderedDict[str, Schema[Component]]) – The blocks to check, in order
Raises: LinkError
– On undeclared blocks (i.e not the right config order)ProtocolError
– Attempt to reduce a non-comparable block
-
flambe.experiment.utils.
check_search
(blocks: Dict[str, Schema], search: Mapping[str, SearchAlgorithm], schedulers: Mapping[str, TrialScheduler])[source]¶ Check validity of links between blocks.
Ensures dependency order, and that only Comparable blocks are being reduced through a LinkBest object.
Parameters: - blocks (OrderedDict[str, Schema[Component]]) – Ordered mapping from block id to a schema of the block
- search (Mapping[str, SearchAlgorithm], optional) – Map from block id to hyperparameter search space generator
- schedulers (Mapping[str, TrialScheduler], optional) – Map from block id to search scheduler
Raises: ProtocolError
– Non computable block assigned a search or scheduler.ProtocolError
– Non comparable block assigned a non default search or scheduler
-
flambe.experiment.utils.
convert_tune
(data: Any)[source]¶ Convert the options and links in the block.
Convert Option objects to tune.grid_search or tune.sample_from functions, depending on the type.
Parameters: data (Any) – Input object that may contain Options objects that should be converted to a Tune-compatible representation
-
flambe.experiment.utils.
traverse
(nested: Mapping[str, Any], path: Optional[List[str]] = None) → Iterable[Any][source]¶ Iterate over a nested mapping returning the path and key, value.
Parameters: - nested (Mapping[str, Any]) – Mapping where some values are also mappings that should be traversed
- path (List[str]) – List of keys that were used to reach the current mapping
Returns: Iterable of path, key, value triples
Return type: Iterable[Any]
-
flambe.experiment.utils.
traverse_all
(obj: Any) → Iterable[Any][source]¶ Iterate over all nested mappings and sequences
Parameters: obj (Any) – Returns: Iterable of child values to obj Return type: Iterable[Any]
-
flambe.experiment.utils.
traverse_spec
(nested: Mapping[str, Any], path: Optional[List[str]] = None) → Iterable[Any][source]¶ Iterate over a nested mapping returning the path and key, value.
Parameters: - nested (Mapping[str, Any]) – Mapping where some values are also mappings that should be traversed
- path (List[str]) – List of keys that were used to reach the current mapping
Returns: Iterable of path, key, value triples
Return type: Iterable[Any]
-
flambe.experiment.utils.
update_nested
(nested: MutableMapping[str, Any], prefix: Iterable[str], key: str, new_value: Any) → None[source]¶ Multi-level set operation for nested mapping.
Parameters: - nested (Mapping[str, Any]) – Nested dictionary where keys are all strings
- prefix (Iterable[str]) – List of keys specifying path to value to be updated
- key (str) – Final key corresponding to value to be updated
- new_value (Any) – New value to set for [p1]…[key] in nested
-
flambe.experiment.utils.
get_nested
(nested: Mapping[str, Any], prefix: Iterable[str], key: str) → Any[source]¶ Get nested value in standard Mapping.
Parameters: - nested (Mapping[str, Any]) – The mapping to index in
- prefix (Iterable[str]) – The path to the final key in the nested input
- key (str) – The key to query
Returns: The value at the given path and key
Return type: Any
-
flambe.experiment.utils.
update_schema_with_params
(schema: Schema, params: Dict[str, Any]) → Schema[source]¶ Replace options in the schema recursivly.
Parameters: - schema (Schema[Any]) – The schema object to update
- params (Dict[str, Any]) – The corresponding nested diciontary with values
Returns: The update schema (same object as the input, not a copy)
Return type: Schema[Any]
-
flambe.experiment.utils.
has_schemas_or_options
(x: Any) → bool[source]¶ Check if object contains Schemas or Options.
Recurses for Mappings and Sequences
Parameters: x (Any) – Input object to check for Schemas and Options Returns: True iff contains any Options or Schemas. Return type: bool
-
flambe.experiment.utils.
divide_nested_grid_search_options
(config: MutableMapping[str, Any]) → Iterable[Mapping[str, Any]][source]¶ Divide config into a config Iterable to remove nested Options.
For every GridSearchOptions or SampledUniformSearchOptions, if any values contain more Options or Schemas, create copies with a single value selected in place of the option. Resulting configs will have no nested options.
Parameters: config (MutableMapping[str, Any]) – MutableMapping (or Schema) containing Options and Schemas Returns: Each Mapping contains variants from original config without nested options Return type: Iterable[Mapping[str, Any]]
-
flambe.experiment.utils.
extract_dict
(config: Mapping[str, Any]) → Dict[str, Any][source]¶ Turn the schema into a dictionary, ignoring types.
NOTE: We recurse if any value is itself a Schema, a Sequence of Schema`s, or a `Mapping of `Schema`s. Other unconvential collections will not be inspected.
Parameters: schema (Schema) – The object to be converted into a dictionary Returns: The output dictionary representation. Return type: Dict
-
flambe.experiment.utils.
extract_needed_blocks
(schemas: Dict[str, Schema], block_id: str, global_vars: Optional[Dict[str, Any]] = None) → Set[str][source]¶ Returns the set of all blocks that the input block links to.
Parameters: - schemas (Dict[str, Schema[Any]]) – Map from block_id to Schema object
- block_id (str) – The block containing links
Returns: The list of ancestor block ids
Return type: List[str]
-
flambe.experiment.utils.
update_link_refs
(schemas: Dict[str, Schema], block_id: str, global_vars: Dict[str, Any]) → None[source]¶ Resolve links in schemas at block_id.
Parameters: - schemas (Dict[str, Schema[Any]]) – Map from block_id to Schema object
- block_id (str) – The block where links should be activated
- global_vars (Dict[str, Any]) – The environment links (ex: resources)
-
flambe.experiment.utils.
get_best_trials
(trials: List[Trial], topk: int, metric='episode_reward_mean') → List[Trial][source]¶ Get the trials with the best result.
Parameters: - trials (List[ray.tune.Trial]) – The list of trials to examine
- topk (int) – The number of trials to reduce to
- metric (str, optional) – The metric used in comparaison (higher is better)
Returns: The list of best trials
Return type: List[ray.tune.Trial]
-
flambe.experiment.utils.
get_non_remote_config
(experiment)[source]¶ Returns a copy of the original config file without the remote configuration
Parameters: experiment (Experiment) – The experiment object
-
flambe.experiment.utils.
local_has_gpu
() → bool[source]¶ Returns is local process has GPU
Returns: Return type: bool
-
flambe.experiment.utils.
rel_to_abs_paths
(d: Dict[str, str]) → Dict[str, str][source]¶ Convert relative paths to absolute paths.
Parameters: d (Dict[str, str]) – A dict from name -> path. Returns: The same dict received as parameter with relative paths replaced with absolute. Return type: Dict[str, str]
-
flambe.experiment.utils.
shutdown_ray_node
() → int[source]¶ Call ‘ray stop’ locally to terminate the ray node.
-
flambe.experiment.utils.
shutdown_remote_ray_node
(host: str, user: str, key: str) → int[source]¶ Execute ‘ray stop’ on a remote machine through ssh to terminate the ray node.
IMPORTANT: this method is intended to be run in the cluster.
Parameters: - host (str) – The Orchestrator’s IP that is visible by the factories (usually the private IP)
- user (str) – The username for that machine.
- key (str) – The key that communicate with the machine.
-
flambe.experiment.utils.
get_default_devices
(debug: bool = False, default_cpus: int = 1, default_gpus: int = 1) → Dict[str, int][source]¶ Get the default devices to use if not provided.
Parameters: - debug (bool, optional) – If we are running in debug mode (where Ray is not available).
If debug is False, this method should be called after running
ray.init()
. - default_cpus (int, optional) – The default number of CPU’s to use. Default
1
. - default_gpus (int, optional) – The default number of GPU’s to use. Default
1
.
Returns: devices – Default set of devices to use. Should have at most two keys: ‘cpu’, and ‘gpu’ if cuda is available.
Return type: Dict[str, int]
Raises: ValueError
– If the total number of GPUs is larger than the number available.- debug (bool, optional) – If we are running in debug mode (where Ray is not available).
If debug is False, this method should be called after running