ModelContainer

class romancal.datamodels.container.ModelContainer(init=None, asn_exptypes=None, asn_n_members=None, iscopy=False, memmap=False, return_open=True, save_open=True)

A container for holding DataModels.

This functions like a list for holding DataModel objects. It can be iterated through like a list and the datamodels within the container can be addressed by index. Additionally, the datamodels can be grouped by exposure.

Parameters:

• init (path to ASN file, list of either datamodels or path to ASDF files, or None) – If None, then an empty ModelContainer instance is initialized, to which datamodels can later be added via the insert(), append(), or extend() method.

• iscopy (bool) – Presume this model is a copy. Members will not be closed when the model is closed/garbage-collected.

• memmap (bool) – Open ASDF file binary data using memmap (default: False)

• return_open (bool) – (optional) See notes below on usage.

• save_open (bool) – (optional) See notes below on usage.

Examples

To load a list of ASDF files into a ModelContainer:

container = ModelContainer(
    [
        "/path/to/file1.asdf",
        "/path/to/file2.asdf",
        ...,
        "/path/to/fileN.asdf"
    ]
)

To load a list of open Roman DataModels into a ModelContainer:

import roman_datamodels.datamodels as rdm
data_list = [
        "/path/to/file1.asdf",
        "/path/to/file2.asdf",
        ...,
        "/path/to/fileN.asdf"
    ]
datamodels_list = [rdm.open(x) for x in data_list]
container = ModelContainer(datamodels_list)

To load an ASN file into a ModelContainer:

asn_file = "/path/to/asn_file.json"
container = ModelContainer(asn_file)

In any of the cases above, the content of each file in a ModelContainer can be accessed by iterating over its elements. For example, to print out the filename of each file, we can run:

for model in container:
    print(model.meta.filename)

Additionally, ModelContainer can be used with context manager:

with ModelContainer(asn_file) as asn:
    # do stuff

Notes

The optional parameters save_open and return_open can be provided to control how the DataModel are used by the ModelContainer. If save_open is set to False, each input DataModel instance in init will be written out to disk and closed, then only the filename for the DataModel will be used to initialize the ModelContainer object. Subsequent access of each member will then open the DataModel file to work with it. If return_open is also False, then the DataModel will be closed when access to the DataModel is completed. The use of these parameters can minimize the amount of memory used by this object during processing.

Warning

Input files will be updated in-place with new meta attribute values when ASN table’s members contain additional attributes.

append(model)

close()

Close all datamodels.

copy(memo=None)

Returns a deep copy of the models in this model container.

property crds_observatory

Get the CRDS observatory for this container. Used when selecting step/pipeline parameter files when the container is a pipeline input.

Return type:

str

extend(input_object)

from_asn(asn_data, asn_file_path=None)

Load ASDF files from a Roman association file.

Parameters:

• asn_data (Association) – Association dictionary.

• asn_file_path (str) – Filepath of the association, if known.

get_crds_parameters()

Get parameters used by CRDS to select references for this model.

Return type:

dict

get_sections()

Iterator to return the sections from all members of the container.

insert(index, model)

merge_tree(a, b)

Merge elements from tree b into tree a.

property models_grouped

Returns a list of a list of datamodels grouped by exposure. Assign an ID grouping by exposure.

Data from different detectors of the same exposure will have the same group id, which allows grouping by exposure. The following metadata is used for grouping:

meta.observation.program meta.observation.observation meta.observation.visit meta.observation.visit_file_group meta.observation.visit_file_sequence meta.observation.visit_file_activity meta.observation.exposure

pop(index=-1)

static read_asn(filepath)

Load ASDF files from a Roman association file.

Parameters:

filepath (str) – The path to an association file.

save(path=None, dir_path=None, save_model_func=None, **kwargs)

Write out models in container to ASDF.

Parameters:

• path (str or func or None) –

  • If None, the meta.filename is used for each model.

  • If a string, the string is used as a root and an index is appended.

  • If a function, the function takes the two arguments: the value of model.meta.filename and the idx index, returning constructed file name.

• dir_path (str) – Directory to write out files. Defaults to current working dir. If directory does not exist, it creates it. Filenames are pulled from meta.filename of each datamodel in the container.

• save_model_func (func or None) – Alternate function to save each model instead of the models save method. Takes one argument, the model, and keyword argument idx for an index.

Note

Additional parameters provided via **kwargs are passed on to roman_datamodels.datamodels.DataModel.to_asdf

Returns:

output_paths – List of output file paths of where the models were saved.

Return type:

[str[, …]]

set_buffer(buffer_size, overlap=None)

Set buffer size for scrolling section-by-section access.

Parameters:

• buffer_size (float, None) – Define size of buffer in MB for each section. If None, a default buffer size of 1MB will be used.

• overlap (int, optional) – Define the number of rows of overlaps between sections. If None, no overlap will be used.