flambe.compile.downloader
¶
Module Contents¶
-
flambe.compile.downloader.
s3_exists
(url: ParseResult) → bool[source]¶ Return is an S3 resource exists.
Parameters: url (ParseResult) – The parsed URL. Returns: True if it exists. False otherwise. Return type: bool
-
flambe.compile.downloader.
s3_remote_file
(url: ParseResult) → bool[source]¶ Check if an existing S3 hosted artifact is a file or a folder.
Parameters: url (ParseResult) – The parsed URL. Returns: True if it’s a file, False if it’s a folder. Return type: bool
-
flambe.compile.downloader.
download_s3_file
(url: str, destination: str) → None[source]¶ Download an S3 file.
Parameters: - url (str) – The S3 URL. Should follow the format: ‘s3://<bucket-name>[/path/to/file]’
- destination (str) – The output file where to copy the content
-
flambe.compile.downloader.
http_exists
(url: str) → bool[source]¶ Check if an HTTP/HTTPS file exists.
Parameters: url (str) – The HTTP/HTTPS URL. Returns: True if the HTTP file exists Return type: bool
-
flambe.compile.downloader.
download_http_file
(url: str, destination: str) → None[source]¶ Download an HTTP/HTTPS file.
Parameters: - url (str) – The HTTP/HTTPS URL.
- destination (str) – The output file where to copy the content. Needs to support binary writing.
-
flambe.compile.downloader.
download_s3_folder
(url: str, destination: str) → None[source]¶ Download an S3 folder.
Parameters: - url (str) – The S3 URL. Should follow the format: ‘s3://<bucket-name>[/path/to/folder]’
- destination (str) – The output folder where to copy the content
-
flambe.compile.downloader.
download_manager
(path: str, destination: Optional[str] = None)[source]¶ Manager for downloading remote URLs
Parameters: - path (str) – The remote URL to download. Currently, only S3 and http/https URLs are supported. In case it’s already a local path, it yields the same path.
- destination (Optional[str]) – The path where the artifact will be downloaded (this includes the file/folder name also). In case of not given, a temporary directory will be used and the name of the artifact will be inferred from the path.
Examples
>>> with download_manager("https://host.com/my/file.zip") as path: >>> os.path.exists(path) >>> True
Yields: str – The new local path