Pipeline

class stpipe.Pipeline(*args, **kwargs)

Bases: Step

A Pipeline is a way of combining a number of steps together.

See Step for the parameters.

Attributes Summary

reference_file_types	Collect the list of all reftypes for child Steps that are not skipped.
spec
step_defs

Methods Summary

get_config_from_reference(dataset[, ...])	Retrieve step parameters from reference database
get_pars([full_spec])	Retrieve the configuration parameters of a pipeline
get_ref_override(reference_file_type)	Return any override for reference_file_type for any of the steps in Pipeline self.
load_spec_file([preserve_comments])
merge_config(config, config_file)
merge_pipeline_config(refcfg, ref_file)	Merge the config parameters from a pipeline config reference file into the config obtained from each step

Attributes Documentation

reference_file_types: ClassVar

Collect the list of all reftypes for child Steps that are not skipped. Overridden reftypes are included but handled normally later by the Pipeline version of the get_ref_override() method defined below.

spec

step_defs: ClassVar = {}

Methods Documentation

classmethod get_config_from_reference(dataset, disable=None, crds_observatory=None)

Retrieve step parameters from reference database

Parameters:

clsstpipe.step.Step

Either a class or instance of a class derived from Step.

datasetstpipe.datamodel.AbstractDataModel or dict

A model of the input file. Metadata on this input file will be used by the CRDS “bestref” algorithm to obtain a reference file. If dict, crds_observatory must be a non-None value.

disable: bool or None

Do not retrieve parameters from CRDS. If None, check global settings.

crds_observatorystr

Observatory name (‘jwst’ or ‘roman’).

Returns:

step_parametersconfigobj

The parameters as retrieved from CRDS. If there is an issue, log as such and return an empty config obj.

get_pars(full_spec=True)

Retrieve the configuration parameters of a pipeline

Parameters are retrieved for the pipeline and all of its component steps.

Parameters:

full_specbool

Return all parameters, including parent-specified parameters. If False, return only parameters specific to the pipeline and steps.

Returns:

parsdict

Keys are the parameters and values are the values.

get_ref_override(reference_file_type)

Return any override for reference_file_type for any of the steps in Pipeline self. OVERRIDES Step.

Returns:

override_filepath or None.

classmethod load_spec_file(preserve_comments=<stpipe.utilities._NotSet object>)

classmethod merge_config(config, config_file)

classmethod merge_pipeline_config(refcfg, ref_file)

Merge the config parameters from a pipeline config reference file into the config obtained from each step

Parameters:

clsjwst.stpipe.pipeline.Pipeline class

The pipeline class

refcfgConfigObj object

The ConfigObj created from crds cfg files from each of the steps in the pipeline

ref_filestring

The name of the pipeline crds step config file

Returns:

ConfigObj of the merged parameters, with those from the pipeline cfg having

precedence over those from the individual steps