flint.utils.data

Dataloader

class flint.utils.data.dataloader.DataLoader(dataset: flint.utils.data.dataset.Dataset, batch_size: Optional[int] = 1, collate_fn: Optional[Callable] = None)

Bases: object

DataLoader provides an iterable over the given dataset. It supports automatic mini-batching now.

Parameters

• dataset (Dataset) – Dataset from which to load the data.

• batch_size (int, optional, default=1) – How many samples per batch to load.

• collate_fn (callable, optional) – Merge a list of samples to form a mini-batch of Tensor(s).

get()

Dataset

class flint.utils.data.dataset.ConcatDataset(datasets: Iterable[flint.utils.data.dataset.Dataset])

Bases: flint.utils.data.dataset.Dataset

Dataset as a concatenation of multiple datasets. This class is useful to assemble different existing datasets.

Parameters

datasets (list) – List of datasets to be concatenated

static cumsum(sequence)

cumulative_sizes: List[int]

datasets: List[flint.utils.data.dataset.Dataset]

class flint.utils.data.dataset.Dataset

Bases: object