Ewoks tasks reference

Each Darfix widget has an underlying Ewoks task that takes care of the processing. The Ewoks tasks can be used independently for headless processing.

The documentation for these Ewoks tasks is given below.

Binning

Rescale images of a Darfix dataset by a given factor.

Identifier:

darfix.tasks.binning.Binning

Task type:

class

Inputs:

dataset* : darfix.dtypes.Dataset

Input dataset containing a stack of images

scale* : float

Factor to rescale images of the dataset.

output_dir : str | ewokscore.missing_data.MissingData= <MISSING_DATA>

Output directory where the binned data will be saved. If not set, use the input dataset directory.

Outputs:

dataset

BlindSourceSeparation

Perform blind source separation on a Darfix dataset. Blind source separation (BSS) comprises all techniques that try to decouple a set of source signals from a set of mixed signals with unknown (or very little) information.

Supported methods are PCA, NICA, NMF and NICA_NMF.

Related article : https://pmc.ncbi.nlm.nih.gov/articles/PMC10161887/#sec3.3.3

Identifier:

darfix.tasks.blindsourceseparation.BlindSourceSeparation

Task type:

class

Inputs:

dataset* : darfix.dtypes.Dataset

Input dataset containing a stack of images

method* : darfix.tasks.blindsourceseparation.Method

Method to use for blind source separation

n_comp : int | ewokscore.missing_data.MissingData= <MISSING_DATA>

Number of components to extract

save : bool | ewokscore.missing_data.MissingData= <MISSING_DATA>

processing_order : int | ewokscore.missing_data.MissingData= <MISSING_DATA>

Outputs:

W

comp

dataset

DataCopy

Identifier:

darfix.tasks.copy.DataCopy

Task type:

class

Inputs:

dataset* : darfix.dtypes.Dataset

Dataset to copy.

Outputs:

dataset

DataPartition

Filter frames with low intensity.

Identifier:

darfix.tasks.datapartition.DataPartition

Task type:

class

Inputs:

dataset* : darfix.dtypes.Dataset

Input dataset containing a stack of images

bins : int | ewokscore.missing_data.MissingData= <MISSING_DATA>

Number of bins to use for partitioning the data. Default is the number of frames in the dataset.

filter_bottom_bin_idx : int | ewokscore.missing_data.MissingData= <MISSING_DATA>

index of the bins to retrieve bottom threshold filter value. If not defined, no filtering is applied.

filter_top_bin_idx : int | ewokscore.missing_data.MissingData= <MISSING_DATA>

index of the bins to retrieve top threshold filter value. If not defined, no filtering is applied.

Outputs:

dataset

DimensionDefinition

Fit dimension of given dataset. If dims are provided then will use them. else will call ‘find_dimensions’ with the provided tolerance or the default one.

Identifier:

darfix.tasks.dimensiondefinition.DimensionDefinition

Task type:

class

Inputs:

dataset* : darfix.dtypes.Dataset

Input dataset containing a stack of images

dims : dict[int, Any] | ewokscore.missing_data.MissingData= <MISSING_DATA>

Dimensions to use for the dataset. If not provided, the task will try to find dimensions from metadata.

Examples:

• {0: {'name': 'diffrx', 'size': 5, 'range': [0.0, 5.0, 1.0]}, 1: {'name': 'diffry', 'size': 10, 'range': [0.0, 10.0, 1.0]}}

tolerance : float | ewokscore.missing_data.MissingData= <MISSING_DATA>

Tolerance to use for finding dimensions from metadata. Default is 1e-9.

is_zigzag : bool | ewokscore.missing_data.MissingData= <MISSING_DATA>

Set to True if the scan was a zigzag scan (slow motor moving back and forth). Defaults to False.

Outputs:

dataset

GrainPlot

Generates and saves maps of Center of Mass, FWHM, Skewness, Kurtosis, Orientation distribution and Mosaicity.

Identifier:

darfix.tasks.grainplot.GrainPlot

Task type:

class

Inputs:

dataset* : darfix.dtypes.Dataset

Input dataset containing a stack of images

dimensions : tuple[int, int]= (0, 1)

Dimension indices to use for the maps. Default is (0, 1), which means the two first dimensions.

range : tuple[Tuple[float, float] | None, Tuple[float, float] | None]= (None, None)

Dimensionrange for the two dimensions. If None, use the Center of Mass min and max for the both dimensions.

save_maps : bool= True

Whether to save the maps to file. Default is True.

filename : str | ewokscore.missing_data.MissingData= <MISSING_DATA>

Only used if save_maps is True. Filename to save the maps to. Default is ‘maps.h5’ in the dataset directory.

orientation_img_origin : Literal['dims', 'center']= dims

Origin for the orientation distribution image. Can be ‘dims’, ‘center’ or None. Default is ‘dims’.

Outputs:

dataset

HDF5DataSelection

Loads data and positioner metadata from a hdf5 file to create a Darfix dataset.

Identifier:

darfix.tasks.hdf5_data_selection.HDF5DataSelection

Task type:

class

Inputs:

raw_input_file* : str

Path to the raw input file.

Examples:

• '/path/to/awesome/file.h5'

raw_detector_data_path* : str

Path to the raw detector data in the input file.

Examples:

• '/1.1/measurement/pco_ff'

raw_metadata_path* : str | ewokscore.missing_data.MissingData

Path to the raw metadata in the input file.

Examples:

• '/1.1/instrument/positioners'

dark_input_file : str | ewokscore.missing_data.MissingData= <MISSING_DATA>

Path to the dark input file. Default is None.

Examples:

• '/path/to/dark/file.h5'

dark_detector_data_path : str | ewokscore.missing_data.MissingData= <MISSING_DATA>

Path to the dark detector data in the input file. Default is None.

Examples:

• '/1.1/measurement/pco_ff'

workflow_title : str | ewokscore.missing_data.MissingData= <MISSING_DATA>

Title of the dataset for display purpose. Empty if not provided.

treated_data_dir : str | ewokscore.missing_data.MissingData= <MISSING_DATA>

Processed output directory. If not provided, will try to find PROCESSED_DATA directory.

Outputs:

dataset

ConcatenateHDF5Scans

Concatenate a set of scans / entries contained in ‘input_file’. If entries_to_concatenate is None then all entries will be concatenated

• ‘detector_data_path’ is the pattern to find all the detector data path. It

is expected to look like ‘{scan}/instrument/measurement/my_detector’. In this case it will look for each scan at the same location. So for the scan ‘1.1’ it will look for ‘1.1/instrument/measurement/my_detector’. For the scan 2.1 it will look for ‘2.1/instrument/measurement/my_detector’… If the {detector} is provided then all the dataset part of the upper path will be browse in order to ‘guess’ the detector to be used. Using the ‘find_detector’ function. * ‘positioners_group_path’ is the pattern to find all positioner groups. And should look like ‘{scan}/instrument/positioners’ For the scan ‘1.1’ it will look for ‘1.1/instrument/positioners’…

Identifier:

darfix.tasks.hdf5_scans_concatenation.ConcatenateHDF5Scans

Task type:

class

Inputs:

input_file* : str

Path to the input file containing scans to concatenate.

Examples:

• '/path/to/input/file.h5'

entries_to_concatenate : tuple[str, ...] | ewokscore.missing_data.MissingData= <MISSING_DATA>

Entries (scans) in the file to concatenate. If not provided, all entries will be concatenated.

Examples:

• ('/1.1', '/2.1', '/3.1')

detector_data_path : str | ewokscore.missing_data.MissingData= <MISSING_DATA>

Path pattern to the detector data in the input file. If {detector} is in the pattern then all the datasets from the subpath will be browsed in order to ‘guess’ the detector to be used. If not provided, {scan}/measurement/{detector} is the default pattern

Examples:

• '{scan}/instrument/measurement/my_detector'
• '{scan}/instrument/measurement/{detector}'

duplicate_detector_frames : bool | ewokscore.missing_data.MissingData= <MISSING_DATA>

If True will avoid creating a VDS for detector frames and create a standard HDF5 dataset. !!! Will duplicate the frames !!!.

positioners_group_path : str | ewokscore.missing_data.MissingData= <MISSING_DATA>

Path pattern to the positioners group in the input file. If not provided, {scan}/instrument/positioners is the default pattern

Examples:

• '{scan}/instrument/positioners'

overwrite : bool | ewokscore.missing_data.MissingData= <MISSING_DATA>

If True then will overwrite the output file if it exists. False if not provided.

output_file : str | ewokscore.missing_data.MissingData= <MISSING_DATA>

Path to the output file. Must be provided if guess_output_file is False.

Examples:

• '/path/to/output/file.h5'

guess_output_file : bool | ewokscore.missing_data.MissingData= <MISSING_DATA>

If True then will try to guess the output file from the input file. False if not provided.

Outputs:

output_file

MetadataTask

‘Place holder’ task for the MetadataWidgetOW. This widget only does display of information an no processing. But having this task allow use to have it compatible with ewoks.

Identifier:

darfix.tasks.metadata.MetadataTask

Task type:

class

Inputs:

dataset*

Outputs:

NoiseRemoval

Apply a list of noise removal operations on a Darfix dataset.

Identifier:

darfix.tasks.noiseremoval.NoiseRemoval

Task type:

class

Inputs:

dataset* : darfix.dtypes.Dataset

Input dataset containing a stack of images

operations : list[dict[str, Any]] | ewokscore.missing_data.MissingData= []

List of noise removal operations to apply to the dataset. Empty list if not provided.”

Available operations :

• ‘Operation.THRESHOLD’: Threshold operation. Parameters: ‘bottom’ (float) and ‘top’ (float). Keep value only if it is between bottom and top.

• ‘Operation.HP’: Hot Pixel removal using median filter operation. Parameters: ‘kernel_size’ (int).

• ‘Operation.BS’: Background subtraction operation. Parameters: ‘method’ (“mean” | “median”) and ‘background_type’ (“Data” | “Unused data (after partition)” | “Dark data”).

• ‘Operation.MASK’: Mask removal operation. Parameters: ‘mask’ (numpy.ndarray 2D containing 0 and 1 where 0 indicates the pixels to be removed).

Examples:

• [{'type': 'THRESHOLD', 'parameters': {'bottom': 10.0, 'top': 1000.0}}, {'type': 'HP', 'parameters': {'kernel_size': 3}}]

copy_dataset : bool= False

If True, operations are applied on a copy of the input dataset. Else, operations are applied directly on the input dataset

Outputs:

dataset

PCA

Compute Principal Component Analysis on a Darfix dataset.

More about PCA : https://en.wikipedia.org/wiki/Principal_component_analysis

Identifier:

darfix.tasks.pca.PCA

Task type:

class

Inputs:

dataset* : darfix.dtypes.Dataset

Input dataset containing a stack of images

num_components : int | ewokscore.missing_data.MissingData= <MISSING_DATA>

Number of principal components to compute.

Outputs:

dataset

vals

Projection

Removes one dimension by projecting (summing) all images in this dimension.

Details in https://gitlab.esrf.fr/XRD/darfix/-/issues/37

Identifier:

darfix.tasks.projection.Projection

Task type:

class

Inputs:

dataset* : darfix.dtypes.Dataset

Input dataset containing a stack of images

dimension* : Sequence[int]

Dimensions indices to project the data onto.

Outputs:

dataset

RockingCurves

Analyze the rocking curve of each pixel of each image of the darfix dataset by fitting to a peak shape, e.g. a Gaussian.

Related article : https://pmc.ncbi.nlm.nih.gov/articles/PMC10161887/#sec3.3.1

Identifier:

darfix.tasks.rocking_curves.RockingCurves

Task type:

class

Inputs:

dataset* : darfix.dtypes.Dataset

Input dataset containing a stack of images

int_thresh : float | None= None

If provided, only the rocking curves with higher ptp (peak to peak) value > int_thresh are fitted, others are assumed to be noise and will be discarded

method : Literal['trf', 'lm', 'dogbox']= trf

Method to use for the rocking curves fit

output_filename : str | pathlib.Path | None= None

Output filename to save the rocking curves results. Results are not saved if not provided

Outputs:

dataset

maps

RoiSelection

Apply a Region of Interest (ROI) selection on a Darfix dataset.

Identifier:

darfix.tasks.roi.RoiSelection

Task type:

class

Inputs:

dataset* : darfix.dtypes.Dataset

Input dataset containing a stack of images.

roi_origin : numpy.ndarray | list[float] | ewokscore.missing_data.MissingData= <MISSING_DATA>

Origin of the ROI to apply. If not provided, dataset will be unchanged.

roi_size : numpy.ndarray | list[float] | ewokscore.missing_data.MissingData= <MISSING_DATA>

Size of the ROI to apply. If not provided, dataset will be unchanged.

Outputs:

dataset

RSMHistogram

Computes Reciprocal Space Map histogram.

Identifier:

darfix.tasks.rsm_histogram.RSMHistogram

Task type:

class

Inputs:

dataset* : darfix.dtypes.Dataset

Input dataset containing a stack of images.

Q* : Tuple[float, float, float]

Scattering vector in oriented pseudocubic coordinates.

a* : float

Pseudocubic lattice parameter.

map_range* : float

Range (in all 3 directions) of the histogram. Center-to-edge distance.

detector* : str

Detector type for the RSM computation.

units : Literal['poulsen', 'gorfman'] | ewokscore.missing_data.MissingData= <MISSING_DATA>

Either ‘poulsen’ [10.1107/S1600576717011037] or ‘gorfman’ [https://arxiv.org/pdf/2110.14311.pdf]. ‘poulsen’ if not provided

n : Tuple[float, float, float] | ewokscore.missing_data.MissingData= <MISSING_DATA>

Surface normal of the sample in oriented pseudocubic hkl

map_shape : Tuple[float, float, float] | ewokscore.missing_data.MissingData= <MISSING_DATA>

Number of bins in each direction.

energy : float | ewokscore.missing_data.MissingData= <MISSING_DATA>

Outputs:

hist_edges

hist_values

ShiftCorrection

Identifier:

darfix.tasks.shiftcorrection.ShiftCorrection

Task type:

class

Inputs:

dataset* : darfix.dtypes.Dataset

Input dataset containing a stack of images

shift : Sequence[float] | Sequence[Sequence[float]] | None= None

Shift to apply to the images. If not provided, dataset will be unchanged.

selected_axis : int | None= None

Selected dimension axis. If not None. We considere a linear shift along this dimension. Darfix convention is : dimension with axis 0 is the fast motor.

copy_dataset : bool= False

If True, operations are applied on a copy of the input dataset. Else, operations are applied directly on the input dataset

Outputs:

dataset

TransformationMatrixComputation

Computes transformation matrix and attach it to the dataset

Identifier:

darfix.tasks.transformation.TransformationMatrixComputation

Task type:

class

Inputs:

dataset* : darfix.dtypes.Dataset

Input dataset containing a stack of images.

kind : Literal['rsm', 'magnification'] | bool | ewokscore.missing_data.MissingData= <MISSING_DATA>

orientation : int | ewokscore.missing_data.MissingData= <MISSING_DATA>

Used only with kind=’magnification’.

magnification : float | ewokscore.missing_data.MissingData= <MISSING_DATA>

To be used only with kind=’magnification’. Magnification factor to apply to the dataset.

pixelSize : Literal['Basler', 'PcoEdge_2x', 'PcoEdge_10x'] | ewokscore.missing_data.MissingData= <MISSING_DATA>

To be used only with kind=’rsm’, distance in micrometers of each pixel.

rotate : bool | ewokscore.missing_data.MissingData= <MISSING_DATA>

To be used only with kind=’rsm’, if True the images with transformation are rotated 90 degrees.

Outputs:

dataset

WeakBeam

Obtain dataset with filtered weak beam and recover its Center of Mass. Save file with this COM for further processing.

Identifier:

darfix.tasks.weakbeam.WeakBeam

Task type:

class

Inputs:

dataset* : darfix.dtypes.Dataset

Input dataset containing a stack of images.

nvalue : float | ewokscore.missing_data.MissingData= <MISSING_DATA>

Increase or decrease the top threshold by this fixed value.

indices : list[int] | ewokscore.missing_data.MissingData= <MISSING_DATA>

Indices of the images to process. If not provided, all images will be processed.

title : str | ewokscore.missing_data.MissingData= <MISSING_DATA>

Title for the output file. If not provided, title is empty.

Outputs:

dataset

ZSum

Sum all images of the dataset or images at specified indices along a given dimension.

Identifier:

darfix.tasks.zsum.ZSum

Task type:

class

Inputs:

dataset* : darfix.dtypes.Dataset

Input dataset containing a stack of images.

selected_axis : int | None= None

Selected axis in dataset.dataset.data. The zsum is computed for each value of this axis.

Outputs:

zsum