jobarchitect.utils

Utilities for jobarchitect.

jobarchitect.utils.mkdir_parents(path)

Create the given directory path.

This includes all necessary parent directories. Does not raise an error if the directory already exists.

Parameters:path – path to create

jobarchitect.utils.output_path_from_hash(dataset_path, hash_str, output_root)

Return absolute output path for a dataset item.

A.k.a. the absolute path to which output data should be written for the datum specified by the given hash.

This function is not responsible for creating the directory.

Parameters:

• dataset_path – path to input dataset
• hash_str – dataset item identifier as a hash string
• output_root – path to output root

Raises:

KeyError if hash string identifier is not in the dataset

Returns:

absolute output path for a dataset item specified by the identifier

jobarchitect.utils.path_from_hash(dataset_path, hash_str)

Return absolute path from a dataset given a hash.

Parameters:

• dataset_path – path to input dataset
• hash_str – dataset item identifier as a hash string

Returns:

absolute path to dataset item

jobarchitect.utils.split_dataset(dataset_path, nchunks)

Return generator yielding lists of file entries.

Parameters:

• dataset_path – path to input dataset
• nchunks – number of chunks the dataset items should be split into

Returns:

generator yielding lists of file entries