datasets

OutputDataset and OutputIterableDataset dataset objects are returned as outputs from Step objects under the output attribute.

To access a column on the dataset objects you can use the __getitem__ operator like so: step.output['column_name']. This will return a OutputDatasetColumn or OutputIterableDatasetColumn column object that can be passed as an input to the inputs argument of a Step.

Tip

You never need to construct a dataset object yourself. They are returned as output from Step objects. If you need to convert in-memory Python data or data in files to a DataDreamer dataset object, see the DataSource steps available in datadreamer.steps.

class datadreamer.datasets.OutputDataset(step, dataset, pickled=False)

Bases: OutputDatasetMixin

property dataset: Dataset

property num_rows: int

__getitem__(key)

Return type:

Any

property column_names: list[str]

property num_columns: int

property step: Step

class datadreamer.datasets.OutputDatasetColumn(step, dataset, pickled=False)

Bases: OutputDatasetColumnMixin, OutputDataset

class datadreamer.datasets.OutputIterableDataset(step, dataset, pickled=False, total_num_rows=None)

Bases: OutputDatasetMixin

property dataset: IterableDataset

property num_rows: None | int

__getitem__(key)

Return type:

Any

class datadreamer.datasets.OutputIterableDatasetColumn(step, dataset, pickled=False, total_num_rows=None)

Bases: OutputDatasetColumnMixin, OutputIterableDataset