Using Custom Code in Flambé¶
While Flambé offers a large number of Component
objects to use in experiments,
researchers will typically need to use their own code, or modify one of our current
component object.
Writing your custom code¶
Flambé configurations support any python object that inherits from Component
.
You can decide to inherit from one of our base classes such as Module
or Dataset
,
or you can inherit from Component
directly.
A Component
must simply implement a run()
method which returns a boolean indicating whether execution
should continue or not (useful for multi-step components such as a Trainer
).
Additionally, if you would like to run hyperparameter search on your custom
component, you must implement the metric()
method which returns the current best metric.
Setting up your extension¶
Flambé offers a simple mechanism to inject custom code in configs. To do so, your code must be wrapped in a pip installable. If you are not familiar with python packages, you can follow these 3 simple steps:
- Make sure all your code is organised in a python module. In the example below,
my_module
is the name of the module containing our custom model and dataset objects.
my_module # The name of your module
├── model.py
├── dataset.py
...
- Next, we wrap the module in another folder, representing our package.
my_package # This is the name of your package
└── my_module # This is the name of your module
├── __init__.py
├── model.py
├── dataset.py
└── ...
- Finally, we write our setup.py file, which will make our package installable. This file is crucial as it indicates the external dependencies of your custom code. Below is a template for your setup file.
from setuptools import setup, find_packages
setup(
name = 'my_package', # the name of your package
version = '1.0.0', # the version of your extension (optional)
packages = find_packages(),
install_requires = ['numpy >= 1.11.1', 'matplotlib >= 1.5.1'], # Dependencies here
)
After you add the setup file to your package, your final folder structure should look like the one below:
my_package
├── setup.py # This file makes your package installable
└── my_module
├── __init__.py
├── model.py
├── dataset.py
└── ...
Using your extension¶
You have built your first extension! You can now use it freely in any configuration,
whether that’d be for an Experiment
,
a Cluster
or any other Runnable
.
To do so, simply add them at the top of your extension, mapping the name of the
module your built (my_module
in the example) to the location of the package
(my_package
in the example).
Important
The name of the module is used as a prefix in your configurations. Not the name of your package.
my_module: path/to/my_package # Must map from module to package path
--- # Note the 3 dashes here
!Experiment
pipeline:
dataset: !my_module.MyDataset # We use the name of your custom module as prefix
Tip
The path to the package may be a local path, a github URL, or the name of package one pypi. The latter allows you to specify a specific version of your extension. For github, we also support links to specific commit or branches.
Flambé will require your extension to be installed. You can do so manually by running:
pip install my_package
or Flambé can install all the extensions specified in your
configuration automatically while executing your config when using the -i
flag:
flambe -i config.yaml