flap.data_object

Created on Tue Jan 22 17:37:32 2019

The DataObject for FLAP and related methods, procedures

@author: Sandor Zoletnik (zoletnik.sandor@ek-cer.hu) Centre for Energy Research

class flap.data_object.DataObject(data_array=None, error=None, data_unit=None, coordinates=None, exp_id=None, data_title=None, info=None, data_shape=None, data_source=None)

Bases: object

Main object in flap used for accessing, modifying and performing calculations with data from various sources.

Parameters:
data_arrayarray_like, optional, default=None

A numpy array containing the data.

errorarray_like, optional, default=None

One or two numpy arrays with the error.

  • One array: Symmetric error.

  • List of two arrays: asymmetric error ([low deviation,high deviation]).

data_unitflap.Unit, optional, default=None

The unit of the data.

coordinatesflap.Coordinate | list of flap.Coordinate, optional, default=None

Coordinates compatible with the data array.

exp_idstr, optional, default=None

Experiment ID. Can be None if not applicable.

data_titlestr, optional, default=None

Description of the data.

infostr, optional, default=None

Information about the DataObject.

data_shapearray_like, optional, default=None

Used only if data_array is not present.

data_sourcestr, optional, default=None

Data sourve description.

abs_value()

Absolute value of data.

Returns:
flap.DataObject
add_coordinate(coordinates=None, data_source=None, exp_id=None, options=None)

A general coordinate conversion interface. Adds the requested coordinate(s) to the data_object.

Parameters:
coordinatesstr | list of str, optional, default=None

List of coordinates to add, identified by unit name.

data_sourcestr, optional, default=None

Date source to override the one contained in the data object.

exp_idstr, optional, default=None

Experiment ID to override the one contained in the data object.

optionsdict, optional, default=None

Dictionary of options to use.

Returns:
flap.DataObject

The modified data object. Note that the input data object remains the same.

add_coordinate_object(coordinate, index=None)

Add a flap.Coordinate instance to the list of coordinates.

Parameters:
coordinateflap.Coordinate

The coordinate to add.

indexint, optional, default=None

The (zero-based) index of the position to insert the new coordinate at. If None, the new coordinate is added to the end of the list.

apsd(coordinate=None, intervals=None, options=None)

Auto-power spectral density calculation. Return a data object with the coordinate replaced by frequency or wavenumber. The power spectrum is calculated in multiple intervals (described by slicing), and the mean and variance will be returned.

Parameters:
coordinatestr, optional, default=None

The name of the coordinate (string) along which to calculate APSD. This coordinate should change only along one data dimension and should be equidistant.

This and all other cordinates changing along the data dimension of this coordinate will be removed. A new coordinate with name ‘Frequency’/’Wavenumber’ will be added. The unit will be derived from the unit of the coordinate (e.g., Hz cm^-1, m^-1).

intervalsdict | str, optional, default=None

Information of processing intervals.

  • If dictionary with a single key: {selection coordinate: description}). Key is a coordinate name which can be different from the calculation coordinate. Description can be Intervals, DataObject or a list of two numbers. If it is a data object with data name identical to the coordinate, the error ranges of the data object will be used for interval. If the data name is not the same as coordinate, a coordinate with the same name will be searched for in the data object and the value_ranges will be used from it to set the intervals.

  • If not a dictionary and not None, it is interpreted as the interval description, the selection coordinate is taken the same as coordinate.

  • If None, the whole data interval will be used as a single interval.

optionsdict, optional, default=None

Options for APSD calculation. (Keys can be abbreviated.) Possible keys and values:

  • ‘Wavenumber’ (default=False):

    • bool: Whether to use 2*Pi*f for the output coordinate scale; this is useful for wavenumber calculation.

  • ‘Resolution’ (default=None):

    • Output resolution in the unit of the output coordinate.

  • ‘Range’ (default=None):

    • Output range in the unit of the output coordinate.

  • ‘Logarithmic’ (default=False):

    • bool: If True, will create logarithmic frequency binning.

  • ‘Interval_n’ (default=8):

    • int: Minimum number of intervals to use for the processing. These are identical length intervals inserted into the input interval list. Default is 8.

  • ‘Error calculation’ (default=True):

    • bool: Whether to calculate the error. Omitting error calculation increases speed. If ‘Interval_n’ is 1, no error calculation is done.

  • ‘Trend removal’ (default=[‘Poly’, 2]): Trend removal description. (See also flap.spectral_analysis._trend_removal().)

    • list:

      • ['poly', n]: Fit an n order polynomial to the data and subtract.

    • str:

      • ‘mean’: Subtract mean.

    • None: Don’t remove trend.

    Trend removal will be applied to each interval separately.

  • ‘Hanning’ (default=True):

    • bool: Whether to use a Hanning window.

Returns:
flap.DataObject
ccf(ref=None, coordinate=None, intervals=None, options=None)

N-dimensional cross-correlation function (CCF) or covariance calculation for the data object, taking ref as reference.

Calculate all CCF between all signals in ref and self, but not inside self and ref. Correlation is calculated along the coordinate(s) listed in coordinate, which should be identical for the to input data objects.

Returns a data object with dimension number self.dim+ref.dim-len(coordinate). The coordinates are replaced by coordinate_name+’ lag’. The CCF is calculated in multiple intervals (described by intervals) and the mean and variance will be returned.

Parameters:
refflap.DataObject, optional, default=None

Reference to use for CPSD calculation. If None, self is used as reference.

coordinatestr | list of str, optional, default=None

The name of the coordinate along which to calculate CCF, or a list of names. Each coordinate should change only along one data dimension and should be equidistant.

This and all other cordinates changing along the data dimension of these coordinates will be removed. New coordinates with coordinate_name+’ lag’ will be added.

intervalsdict | str, optional, default=None

Information of processing intervals.

  • If dictionary with a single key: {selection coordinate: description}). Key is a coordinate name which can be different from the calculation coordinate. Description can be Intervals, DataObject or a list of two numbers. If it is a data object with data name identical to the coordinate, the error ranges of the data object will be used for interval. If the data name is not the same as coordinate, a coordinate with the same name will be searched for in the data object and the value_ranges will be used from it to set the intervals.

  • If not a dictionary and not None, it is interpreted as the interval description, the selection coordinate is taken the same as coordinate.

  • If None, the whole data interval will be used as a single interval.

optionsdict, optional, default=None

Options for CCF calculation. (Keys can be abbreviated.) Possible keys and values:

  • ‘Resolution’ (default=None):

    • Output resolution for each coordinate. (Single value or list of values.)

  • ‘Range’ (default=None):

    • Output ranges for each coordinate. (List or list of lists.)

  • ‘Interval_n’ (default=8):

    • int: Minimum number of intervals to use for the processing. These are identical length intervals inserted into the input interval list. Default is 8.

  • ‘Error calculation’ (default=True):

    • bool: Whether to calculate the error. Omitting error calculation increases speed. If ‘Interval_n’ is 1, no error calculation is done.

  • ‘Trend removal’ (default=[‘Poly’, 2]): Trend removal description. (See also flap.spectral_analysis._trend_removal().)

    • list:

      • ['poly', n]: Fit an n order polynomial to the data and subtract.

    • str:

      • ‘mean’: Subtract mean.

    • None: Don’t remove trend.

    Trend removal will be applied to each interval separately.

  • ‘Normalize’ (default=False):

    • bool: whether to normalize with autocorrelations, that is, calculate correlation instead of covariance.

  • ‘Verbose’ (default=True):

    • bool: Whether to print progress messages.

Returns:
flap.DataObject
check()

Perform a consistency check for the data object and raise errors if problems are found.

coordinate(name, index=None, options=None)

Return the coordinates of a subarray of the data array.

Parameters:
namestr

Coordinate (unit) name.

indextuple of objects, optional, default=None

The indexes into the data array. (For more details, see flap.coordinate.Coordinate.data().)

options_type_, optional, default=None

The same options as for flap.coordinate.Coordinate.data().

Returns:
datanp.ndarray

The coordinates. The number of dimension is the same as the dimension of the data array, but the number of elements are taken from index.

data_lownp.ndarray

Low ranges, same shape as data. None if no range data is present.

data_highnp.ndarray

High ranges, same shape as data. None if no range data is present.

Warning

Not to be confused with DataObject.coordinates.

coordinate_change_dimensions(name)

Return the list of dimensions of the data array along which the named named coordinate changes.

Parameters:
namestr

Coordinate (unit) name.

Returns:
list
coordinate_change_indices(name)

Return the indices to the data array for which the coordinate changes.

Parameters:
namestr

Coordinate (unit) name.

Returns:
tuple of int

The number of elements in the tuple returned equals the dimension of the data. This can be directly used to get the coordinate values using coordinate().

coordinate_names()

Return a list with the coordinate names.

Returns:
list
coordinate_nochange_dimensions(name)

Return the list of dimensions of the data array along which this coordinate does not change.

Parameters:
namestr

Coordinate (unit) name.

Returns:
list
coordinate_range(name, index=Ellipsis)

Return the data range and the data range with errors for the coordinate.

Parameters:
namestr

Coordinate (unit) name.

indexobject, optional, default=Ellipsis

Unused argument.

Returns:
value_range, value_range_errorslist
property coordinates

Coordinates of the data object.

Returns:
flap.Coordinate | list of flap.Coordinate

Warning

Not to be confused with DataObject.coordinate().

cpsd(ref=None, coordinate=None, intervals=None, options=None)

Complex cross-power spectral density (CPSD) calculation.

Calculate all spectra between all signals in ref and self, but not inside self and ref. Objects self and ref should both have the same equidistant coordinate with equal sampling points.

Returns a data object with dimension number self.dim+ref.dim-1. The coordinate is replaced by frequency or wavenumber. The spectrum is calculated in multiple intervals (described by intervals) and the mean and variance will be returned.

Parameters:
refflap.DataObject, optional, default=None

Reference to use for CPSD calculation. If None, self is used as reference.

coordinatestr, optional, default=None

The name of the coordinate along which to calculate CPSD. This coordinate should change only along one data dimension and should be equidistant.

This and all other cordinates changing along the data dimension of this coordinate will be removed. A new coordinate with name ‘Frequency’/’Wavenumber’ will be added. The unit will be derived from the unit of the coordinate (e.g., Hz cm^-1, m^-1).

intervalsdict | str, optional, default=None

Information of processing intervals.

  • If dictionary with a single key: {selection coordinate: description}). Key is a coordinate name which can be different from the calculation coordinate. Description can be Intervals, DataObject or a list of two numbers. If it is a data object with data name identical to the coordinate, the error ranges of the data object will be used for interval. If the data name is not the same as coordinate, a coordinate with the same name will be searched for in the data object and the value_ranges will be used from it to set the intervals.

  • If not a dictionary and not None, it is interpreted as the interval description, the selection coordinate is taken the same as coordinate.

  • If None, the whole data interval will be used as a single interval.

optionsdict, optional, default=None

Options for APSD calculation. (Keys can be abbreviated.) Possible keys and values:

  • ‘Wavenumber’ (default=False):

    • bool: Whether to use 2*Pi*f for the output coordinate scale; this is useful for wavenumber calculation.

  • ‘Resolution’ (default=None):

    • Output resolution in the unit of the output coordinate.

  • ‘Range’ (default=None):

    • Output range in the unit of the output coordinate.

  • ‘Logarithmic’ (default=False):

    • bool: If True, will create logarithmic frequency binning.

  • ‘Interval_n’ (default=8):

    • int: Minimum number of intervals to use for the processing. These are identical length intervals inserted into the input interval list. Default is 8.

  • ‘Error calculation’ (default=True):

    • bool: Whether to calculate the error. Omitting error calculation increases speed. If ‘Interval_n’ is 1, no error calculation is done.

  • ‘Trend removal’ (default=[‘Poly’, 2]): Trend removal description. (See also flap.spectral_analysis._trend_removal().)

    • list:

      • ['poly', n]: Fit an n order polynomial to the data and subtract.

    • str:

      • ‘mean’: Subtract mean.

    • None: Don’t remove trend.

    Trend removal will be applied to each interval separately.

  • ‘Hanning’ (default=True):

    • bool: Whether to use a Hanning window.

  • ‘Normalize’ (default=False):

    • bool: Whether to use normalization.

Returns:
flap.DataObject
del_coordinate(name)

Delete a coordinate by its unit name.

Parameters:
namestr

Unit name of the coordinate to delete.

detrend(coordinate=None, intervals=None, options=None)

Trend removal.

Parameters:
coordinatestr, optional, default=None

The ‘x’ coordinate for the trend removal.

intervalsdict | str, optional, default=None

Information of processing intervals.

  • If dictionary with a single key: {selection coordinate: description}). Key is a coordinate name which can be different from the calculation coordinate. Description can be Intervals, DataObject or a list of two numbers. If it is a data object with data name identical to the coordinate, the error ranges of the data object will be used for interval. If the data name is not the same as coordinate, a coordinate with the same name will be searched for in the data object and the value_ranges will be used from it to set the intervals.

  • If not a dictionary and not None, it is interpreted as the interval description, the selection coordinate is taken the same as coordinate.

  • If None, the whole data interval will be used as a single interval.

optionsdict, optional, default=None

Options for detrending. (Keys can be abbreviated.) Possible keys and values:

  • ‘Trend removal’ (default=[‘Poly’, 2]): Trend removal description. (See also flap.spectral_analysis._trend_removal().)

    • list:

      • ['poly', n]: Fit an n order polynomial to the data and subtract.

    • str:

      • ‘mean’: Subtract mean.

    • None: Don’t remove trend.

Trend removal will be applied to each interval separately.

Returns:
flap.DataObject
error_value(options=None)

Return a DataObject containing the error of the data.

Parameters:
optionsdict, optional, default=None

Possible keys and values:

  • ‘High’ (default=True):

    • bool: Use high error if error is asymmetric.

  • ‘Low’ (default=False):

    • bool: Use low error is error is asymmetric

Returns:
flap.DataObject
filter_data(coordinate=None, intervals=None, options=None)

1D data filter.

Parameters:
coordinatestr, optional, default=None

The ‘x’ coordinate for the filtering.

intervalsdict | str, optional, default=None

Information of processing intervals.

  • If dictionary with a single key: {selection coordinate: description}). Key is a coordinate name which can be different from the calculation coordinate. Description can be Intervals, DataObject or a list of two numbers. If it is a data object with data name identical to the coordinate, the error ranges of the data object will be used for interval. If the data name is not the same as coordinate, a coordinate with the same name will be searched for in the data object and the value_ranges will be used from it to set the intervals.

  • If not a dictionary and not None, it is interpreted as the interval description, the selection coordinate is taken the same as coordinate.

  • If None, the whole data interval will be used as a single interval.

optionsdict, optional, default=None

Options for filtering. Possible keys and values:

  • ‘Type’ (default=None):

    • None: Do nothing.

    • ‘Int’: Single-term IIF filter, like RC integrator.

    • ‘Diff’: Single term IIF filter, like RC differentiator.

    • ‘Bandpass’ | ‘Lowpass’ | ‘Highpass’: Filters designed by scipy.signal.iirdesign. The filter type is in ‘Design’. Relation of filters and required cutting frequencies:

      • Bandpass: ‘f_low’ – ‘f_high’

      • Lowpass: – ‘f_high’

      • Highpass: – ‘f_low’

  • ‘Design’ (default=’Elliptic’):

    • ‘Elliptic’ | ‘Butterworth’ | ‘Chebyshev I’ | ‘Chebyshev II’ | ‘Bessel’: The design type of the bandpass, lowpass or highpass filter.

      The numpy.iirdesign function is used for generating the filter.

      Setting inconsistent parameters can cause strange results. E.g. too high attenuation at too low frequency relative to the smapling frequency can be a problem.

  • ‘f_low’ (default=None), ‘f_high’ (default=None):

    • float: Cut on/off frequencies in Hz. (Middle between passband and stopband edge.)

  • ‘Steepness’ (default=0.2):

    • float: Difference between passband and stopband edge frequencies as a fraction of the middle frequency.

  • ‘Loss’ (default=3):

    • float: The maximum loss in the passband in dB.

  • ‘Attenuation’ (default=20):

    • float: The minimum attenuation in the stopband dB.

  • ‘Tau’ (default=None):

    • Time constant for integrator/differentiator (in units of the coordinate).

  • ‘Power’ (default=False):

    • bool: Whether to calculate square of the signal after filtering.

  • ‘Inttime’ (default=None):

    • Integration time after power calculation (in units of coordinate).

Returns:
flap.DataObject
get_coordinate_object(name)

Return the Coordinate class having the given name. The returned object is a reference (link), not a copy.

Parameters:
namestr

The name of the coordinate object to get.

Returns:
flap.Coordinate
imag()

Imaginary part of the data.

Has no effect on real-valued data.

Returns:
flap.DataObject
index_from_coordinate(name, coord_values)

Return the closest data indices of the coordinate values.

Coordinates should change only along one dimension. It is assumed that coordinates change monotonically.

Parameters:
namestr

Coordinate name.

coord_valuesarray_like

The coordinate values to convert to indices.

Returns:
array_like

Indices in the same format as coord_values.

pdf(coordinate=None, intervals=None, options=None)

Amplitude distribution function (PDF) of data.

Flattens the data array in the dimensions where the coordinates change and calculates PDF on this data for each case of the other dimensions.

Parameters:
coordinatestr | list of str, optional, default=None

The name of the coordinate(s) along which to calculate. If not set, the first coordinate in the data object will be used. These coordinates will be removed and replaced by a new coordinate with the name of the data.

intervalsdict | str, optional, default=None

Information of processing intervals.

  • If dictionary with a single key: {selection coordinate: description}). Key is a coordinate name which can be different from the calculation coordinate. Description can be Intervals, DataObject or a list of two numbers. If it is a data object with data name identical to the coordinate, the error ranges of the data object will be used for interval. If the data name is not the same as coordinate, a coordinate with the same name will be searched for in the data object and the value_ranges will be used from it to set the intervals.

  • If not a dictionary and not None, it is interpreted as the interval description, the selection coordinate is taken the same as coordinate (or its first element).

  • If None, the whole data interval will be used as a single interval.

optionsdict, optional, default=None

Options for generating the PDF. Possible keys and values:

  • ‘Range’ (default=None):

    • list of float: The data value range. If not set, the data minimum-maximum will be used.

  • ‘Resolution’ (default=None):

    • float: The resolution of the PDF.

  • ‘Number’ (default=None):

    • The number of intervals in ‘Range’. This is an alternative to ‘Resolution’.

Returns:
flap.DataObject
phase()

Phase of data.

Returns:
flap.DataObject
plot(axes=None, slicing=None, slicing_options=None, summing=None, options=None, plot_type=None, plot_options=None, plot_id=None)

Plot a data object.

Parameters:
axeslist of str, optional, default=None

A list of coordinate names. They should be one of the coordinate names in the data object or ‘__Data__’. They describe the axes of the plot. If the number of axes is less than the number required for the plot, ‘__Data__’ will be added. If no axes are given, the default will be used, depending on the plot type. E.g. for an x-y plot the default first axis is the first coordinate, the second axis is ‘__Data__’.

slicingdict, optional, default=None

Passed as an argument to slice_data(). Slicing will be applied before plotting.

slicing_optionsdict, optional, default=None

Options for slicing. See slice_data() for details.

summingstr, optional, default=None

Passed as an argument to slice_data(). Slicing will be applied before plotting.

optionsdict, optional, default=None

Options for plotting. Possible keys and values:

  • ‘Error’ (default=True):

    • True: Plot all error bars

    • False: Do not plot errors.

    • int > 0: Plot this many error bars in plot.

  • ‘Y separation’ (default=None):

    • float: Vertical separation of curves in multi xy plot. For linear scale this will be added to consecutive curves. For Log scale, consecutive curves will be multiplied by this.

  • ‘Log x’ (default=False):

    • bool: Whether to use logscale X axis.

  • ‘Log y’ (default=False):

    • bool: Whether to use logscale Y axis.

  • ‘Log z’ (default=False):

    • bool: Whether to use logscale Z axis.

  • ‘All points’ (default=False):

    • bool: Default is False. If True, will plot all points, otherwise will plot only a reduced number of points (see ‘Maxpoints’). Each plotted point will be the mean of data in a box, and a vertical bar will indicate the value range in each box.

  • ‘Maxpoints’ (default=4000):

    • int: The maximum number of data points plotted. Above this, only this many points will be plotted if ‘All points’ is False.

  • ‘Complex mode’ (default=’Amp-phase’):

    • ‘Amp-phase’: Plot amplitude and phase.

    • ‘Real-imag’: Plot real and imaginary part.

  • ‘X range’,’Y range’ (default=None):

    • lists of two floats: Axes ranges.

  • ‘Z range’ (default=None):

    • list of two floats: Range of the vertical axis.

  • ‘Colormap’ (default=None):

    • str: Cmap name for image and contour plots.

  • ‘Levels’ (default=10):

    • int: Number of contour levels or array of levels.

  • ‘Aspect ratio’ (default=’auto’):

    • ‘equal’, ‘auto’ or float. (See imshow.)

  • ‘Waittime’ (default=1):

    • float: Time to wait in seconds between two images in anim-… type plots

  • ‘Video file’ (default=None):

    • str: Name of output video file for anim-… plots

  • ‘Video framerate’ (default=20):

    • float: Frame rate for output video.

  • ‘Video format’ (default=’avi’):

    • str: Format of the video. Valid options: ‘avi’.

  • ‘Clear’ (default=False):

    • bool: If True, don’t use the existing plots, generate new ones. (No overplotting.)

  • ‘Force axes’ (default=False):

    • True: Force overplotting, even if axes are incpomatible.

  • ‘Colorbar’ (default=True):

    • bool: Switch on/off colorbar

  • ‘Nan color’ (default=None):

    • The color to use in image data plotting for np.nan (not-a-number) values.

  • ‘Interpolation’ (default=’bilinear’):

    • Interpolation method for image plot.

  • ‘Language’ (default=’EN’):

    • str: Language of certain standard elements in the plot. Possible values: {‘EN’, ‘HU’}.

  • ‘EFIT options’ (default=None):

    • Dictionary of EFIT plotting options:

      • ‘Plot separatrix’: Set this to plot the separatrix onto the video.

      • ‘Separatrix X’: Name of the flap.DataObject for the separatrix X data (usually ‘R’).

      • ‘Separatrix Y’: Name of the flap.DataObject for the separatrix Y data (usually ‘z’).

      • ‘Separatrix XY’: Name of the 2D flap.DataObject for the separatrix XY data (usually ‘Rz’).

      • ‘Separatrix color’: Color of the separatrix for the plot.

      • ‘Plot limiter’: Set to plot the limiter onto the video.

      • ‘Limiter X’: Name of the flap.DataObject for the limiter X data (usually ‘R’).

      • ‘Limiter Y’: Name of the flap.DataObject for the limiter Y data (usually ‘z’).

      • ‘Limiter XY’: Name of the 2D flap.DataObject for the limiter XY data (usually ‘Rz’).

      • ‘Limiter color’: Color of the limiter for the plot.

      • ‘Plot flux surfaces’: Name of the 2D flap.DataObject for the flux surfaces (should have the same coordinate names as the plot).

      • ‘nlevels’: Number of contour lines for the flux surface plotting.

  • ‘Prevent saturation’ (default=False):

    • bool: Prevent saturation of the video signal when it exceeds zrange[1]. It uses data modulo zrange[1] ito overcome the saturation. (Works for animation.)

plot_type{‘xy’, ‘multi xy’, ‘grid xy’, ‘image’, ‘contour’, ‘anim-image’, ‘anim-contour’}, optional, default=None

The plot type. Can be abbreviated. Possible values:

  • ‘xy’: Simple 1D plot. Default axes are: first coordinate, ‘__Data__’.

  • ‘multi xy’: In case of 2D data, plots 1D curves with a vertical shift. Default x axis is the first coordinate, y axis is ‘Data’. The signals are labeled with the ‘Signal name’ coordinate, or with the one named in options['Signal name'].

  • ‘grid xy’: In case of 3D data, plots xy plots on a 2D grid. Axes are: grid x, grid y, x, y All coordinates should change in one dimension.

  • ‘image’: Plots a 2D data matrix as an image. Options: ‘Colormap’, ‘Data range’, …

  • ‘contour’: Contour plot.

  • ‘anim-image’, ‘anim-contour’: Like ‘image’ and ‘contour’, but the third axis is time.

plot_optionsdict | list of dict, optional, default=None

Dictionary containg matplotlib plot options. Will be passed over to the plot call. For plots with multiple subplots, this can be a list of dictionaries, each for one subplot.

plot_idflap.plot.PlotID, optional, default=None

Can be used for overplotting, if the new plot should go into an existing plot.

Returns:
flap.plot.PlotID

The plot ID of the current plot. This can be used later for overplotting.

proc_interval_limits(coordinate, intervals=None)

Determine processing interval limits, both in coordinates and data indices.

This is a helper routine for all functions which do some calculation as a function of one coordinate and allow processing only a set of intervals instead of the whole dataset.

Parameters:
coordinatestr

Name of the coordinate along which calculation will be done. This must change only along a single data dimension.

intervalsdict | str, optional, default=None

Information of processing intervals.

  • If dictionary with a single key: {selection coordinate: description}). Key is a coordinate name which can be different from the calculation coordinate. Description can be Intervals, DataObject or a list of two numbers. If it is a data object with data name identical to the coordinate, the error ranges of the data object will be used for interval. If the data name is not the same as coordinate, a coordinate with the same name will be searched for in the data object and the value_ranges will be used from it to set the intervals.

  • If not a dictionary and not None, it is interpreted as the interval description, the selection coordinate is taken the same as coordinate.

  • If None, the whole data interval will be used as a single interval.

Returns:
calc_int, calc_int_ind, sel_int, sel_int_ind: list of np.ndarray

Each return value is a list of numpy arrays: [start, end]. The calc_xxx values are for the calculation coordinate, the sel_xxx are for the selection coordinate. xxx_int is in coordinate values, xxx_int_ind is in data index values, following the Python convention, as the end index is not included. The index start indices will be always smaller than the end indices.

real()

Real part of the data.

Has no effect on real-valued data.

Returns:
flap.DataObject
regenerate_coordinates()

Regenerate coordinates.

See slice_data() for details.

remove_sharp_peaks(coordinate=None, options=None)

Remove sharp peaks with only a few samples of duration from the data, e.g. due to neutron/gamma noise affecting APDCAM.

Peaks are detected by looking for sharp (as specified by the diff_limit option) rising and falling edges in the signal, within a window of maximum max_width_samples samples. Detected peaks can be removed from the signal by interpolation (see remove_peaks below).

By default, only a single data object is returned, with the number of removed/detected peaks and number of samples belonging to peaks added to the info field of the data object if return_num_detected_peaks is True. The keys added are: ‘num_peak_samples_detected’, ‘num_peaks_detected’, ‘num_peak_samples_removed’, ‘num_peaks_removed’.

If return_loc_detected_samples is set to True, an additional coordinate named ‘Peak detected’ is added to the data object, indicating whether a sample belongs to a detected peak.

If return_peak_shapes is set to True, the center times and the shapes (width, amplitude) of the detected peaks are also returned as two additional data objects (separately for each time series, if applicable). These data objects contain the center times of the peaks as a non-equidistant coordinate axis, with the width/amplitude of the peaks as the data.

Parameters:
coordinatestr, optional, default=None

The name of the coordinate along which to remove peaks. If not set, a coordinate named ‘Time’ in the DataObject will be used, if there is one. Must be equidistant and change along one dimension only. If the data changes along other dimensions as well, the peaks will be detected/removed separately for each slice along the other dimensions.

optionsdict, optional, default=None

Options for peak removal. Possible keys and values:

  • ‘max_width_samples’:

    • int: Maximum width of a peak in samples.

  • ‘diff_limit’:

    • The derivative limit for the detection of sharp rising/falling edges of peaks. Calculated in units of (signal units) / (time units).

  • ‘return_loc_detected_samples’ (default=False):

    • bool: Whether to return an additional coordinate axis indicating if a sample belongs to a detected/removed peak.

  • ‘return_num_detected_peaks’ (default=False):

    • bool: Whether to return the number of detected peaks.

  • ‘return_peak_shapes’ (default=False):

    • bool: Whether to return the widths and amplitudes of detected peaks as two additional data objects.

  • ‘remove_peaks’ (default=True):

    • bool: Whether to remove the detected peaks.

  • ‘interpolation’ (default=’linear’):

    • ‘linear’ : Interpolation method for removing peaks.

Returns:
flap.DataObject | (flap.DataObject, array of flap.DataObject, array of flap.DataObject)

If return_peak_shapes is False, a single data object is returned (with the peaks removed, if requested). If return_peak_shapes is True, a tuple of three data objects is returned: besides the one returned by default, the second containing the peak widths, and the third containing the peak amplitudes.

save(filename, protocol=3)

Save the data object to a binary file.

Saving uses the pickle protocol. Use load() to read the object.

Parameters:
filenamestr

The name of the output file.

protocolconst

The pickle protocol to use.

slice_data(slicing=None, summing=None, options=None)

Slice (select areas) from the data object along one or more coordinates.

Parameters:
slicingdict, optional, default=None

Dictionary with keys referring to coordinates in the data object. Values can be:

  1. For SIMPLE SLICE: cases when closest value or interpolated value is selected.

    Possibilities:

    1. slice objects, range objects, scalars, lists, numpy array.

    2. DataObject objects without error and with data unit.name equal to the coordinate.

    3. DataObject with the name of one coordinate equal to the dictionary key without having value_ranges values.

    4. Intervals objects with one interval.

  2. For MULTI SLICE: Various range selection objects. In this case ranges are selected and a new dimension is added to the data array (only if more than 1 interval is selected) going through the intervals. If intervals are of different length, the longest will be used and missing elements filled with float('nan').

    Two new coordinates are added: “<coordinate> in interval”, “<coordinate> interval”.

    Possibilities:

    1. Intervals objects with more than one interval

    2. DataObject objects with data unit.name equal to the slicing coordinate. The error values give the intervals.

    3. DataObject with the name of one coordinate equal to the slicing coordinate. The value_ranges select the intervals.

    If range slicing is done with multiple coordinates which have common element in the dimension list, they will be done in one step. Otherwise the slicing is done sequentially.

summingdict, optional, default=None

Summing is applied to the sliced data. It processes data along one coordinate and the result is a scalar. This way summing reduces the number of dimensions.

Dictionary with keys referring to coordinates and values as processing strings. If the processed coordinate changes along multiple dimensions, those dimensions will be flattened.

For mean and avereage data, errors are calculated as error of independent variables. That is, taking the square root of the squared sum of errors. For coordinates the mean of the ranges is taken.

Processing strings are the following:

  • None: Nothing to be done in this dimension.

  • ‘Mean’: take mean of values in selection/coordinate.

  • ‘Sum’: take sum of values in selection/coordinate.

  • ‘Min’: take the minimum of the values/coordinate.

  • ‘Max’: take the maximum of the values/coordinate.

optionsdict, optional, default=None

Possible keys and values:

  • ‘Partial intervals’ (default=True):

    • bool: If True, processes intervals which extend over the coordinate limits. If False, only full intervals are processed.

  • ‘Slice type’ (default=None):

    • ‘Simple’: Case a) above: closest or interpolated values are selected, dimensions are reduced or unchanged.

    • ‘Multi’: Case b) above: multiple intervals are selected, and their data is placed into a new dimension.

    • None: Automatically select. For slicing data in case b multi slice, otherwise simple

  • ‘Interpolation’ (default=’Closest value’):

    • ‘Closest value’

    • ‘Linear’

  • ‘Regenerate coordinates’ (default=True): - bool: Default is True. If True, and summing is done, then looks for pairs of coordinates (‘Rel. <coord> in int(<coord1>)’, ‘Start <coord> in int(<coord1>)’).

    If such pairs are found, and they change on the same dimension or one of them is constant, then coordinate <coord> is regenerated and these are removed.

Returns:
flap.DataObject

The sliced object.

slicing_to_intervals(slicing)

Convert a multi-slicing description to an Intervals object. For possibilities, see slice_data().

Parameters:
slicingdict

The slicing to use.

Returns:
flap.Intervals
stft(coordinate=None, options=None)

Calculate the short-time Fourier transform (STFT) of the data using scipy.signal.stft.

Parameters:
coordinatestr, optional, default=None

The name of the coordinate along which to calculate the STFT. This coordinate should change only along one data dimension and should be equidistant.

This and all other coordinates changing along the data dimension of this coordinate will be removed. A new coordinate with name ‘Frequency’ (unit Hz) will be added.

optionsdict, optional, default=None

Options of STFT will be passed to scipy.signal.stft.

Returns:
flap.DataObject
to_intervals(coordinate)

Create an Intervals class object from either the data error ranges or the coordinate value ranges.

Parameters:
coordinatestr

Coordinate (unit) name.

Returns:
flap.coordinates.Intervals
class flap.data_object.FlapStorage

Bases: object

Stores data and data source information.

add_coord_functions()

Return the stored add_coord functions.

Returns:
list
add_data_object(d, name)

Add a data object to the storage.

Parameters:
dflap.DataObject

The data object to add.

namestr

Name of the data object.

add_data_source(data_source, get_data_func=None, add_coord_func=None)

Add data source.

Parameters:
data_sourcestr

Name of the data source to add.

get_data_funccallable, optional, default=None

The function used for retrieving data.

add_coord_funccallable, optional, default=None

The function used for adding coordinates.

data_objects()

Return the stored data objects.

Returns:
dict
data_sources()

List of data sources.

Returns:
list
delete_data_object(name, exp_id=None)

Delete a data object.

Parameters:
namestr

The name of the data object to be deleted.

exp_idstr, optional, default=None

Experiment ID. Supports extended Unix regular expressions.

find_data_objects(name, exp_id='*')

Find data objects in storage.

Parameters:
namestr

Name of data object.

exp_idstr, optional, default=’*’

Experiment ID. Supports extended Unix regular expressions.

Returns:
name_listlist

List of names of objects found.

exp_ID_listlist

List of experiment IDs of objects found.

get_data_functions()

Return the stored get_data functions.

Returns:
list
get_data_object(name, exp_id='*')

Retrieve an object from storage.

Note that the returned object is a copy, not a reference.

Parameters:
namestr

Name of the object to retrieve.

exp_idstr, optional, default=’*’

Experiment ID. Supports extended Unix regular expressions.

Returns:
flap.DataObject
get_data_object_ref(name, exp_id='*')

Retrieve a reference to an object from storage.

Note that the returned object is a reference, not a copy.

Parameters:
namestr

Name of the object to retrieve.

exp_idstr, optional, default=’*’

Experiment ID. Supports extended Unix regular expressions.

Returns:
flap.DataObject
list_data_objects(name, exp_id='*')

List data objects in storage.

Parameters:
namestr | list of flap.DataObject

Name (with wildcards), or list of data objects.

exp_idstr, optional, default=’*’

Experiment ID. Supports extended Unix regular expressions.

Returns:
d_listlist

List of data objects.

nameslist

List of corresponding names.

expslist

List of corresponding experiment IDs.

flap.data_object.abs_value(object_name, exp_id='*', output_name=None)

Absolute value of the data contained in the DataObject retrieved from flap_storage.

Parameters:
object_namestr

Name identifying the data object in flap_storage.

exp_idstr, optional, default=’*’

Experiment ID. Supports extended Unix regular expressions.

output_namestr, optional, default=None

Name of the new data object added to flap_storage.

Returns:
flap.DataObject
flap.data_object.add_coordinate(object_name, coordinates=None, exp_id='*', options=None, output_name=None)

Add coordinates to a data object ib flap_storage.

Parameters:
object_namestr

Name of data object.

coordinatesstr | list of str, optional, default=None

List of coordinates to add, identified by unit name.

exp_idstr, optional, default=’*’

Experiment ID. Supports extended Unix regular expressions.

optionsdict, optional, default=None

Possible keys and values:

  • ‘exp_ID’:

    • str: Use this exp_id for calculating coordinates instead of the one in the data object

  • ‘data_source’:

    • str: Use this data source instead of the one in the data object.

Other elements of options are passed on to flap.data_object.DataObject.add_coordinate().

output_namestr, optional, default=None

The name of the new data object in the storage. If None, the input will be overwritten.

Returns:
flap.DataObject
flap.data_object.add_data_object(d, object_name)

Add a data object to flap_storage.

Parameters:
dflap.DataObject

The data object to add.

object_namestr

Name of the data object.

flap.data_object.apsd(object_name, exp_id='*', output_name=None, coordinate=None, intervals=None, options=None)

Auto-power spectral density calculation for a data object in flap_storage. Returns a data object with the coordinate replaced by frequency or wavenumber. The power spectrum is calculated in multiple intervals (described by slicing), and the mean and variance will be returned.

Parameters:
object_namestr

Name identifying the data object in flap_storage.

exp_idstr, optional, default=’*’

Experiment ID. Supports extended Unix regular expressions.

output_namestr, optional, default=None

Name of the new data object added to flap_storage.

coordinatestr, optional, default=None

The name of the coordinate along which to calculate APSD. This coordinate should change only along one data dimension and should be equidistant.

This and all other cordinates changing along the data dimension of this coordinate will be removed. A new coordinate with name ‘Frequency’/’Wavenumber’ will be added. The unit will be derived from the unit of the coordinate (e.g., Hz cm^-1, m^-1).

intervalsdict | str, optional, default=None

Information of processing intervals.

  • If dictionary with a single key: {selection coordinate:

    description}). Key is a coordinate name which can be different from the calculation coordinate. Description can be Intervals, DataObject or a list of two numbers. If it is a data object with data name identical to the coordinate, the error ranges of the data object will be used for interval. If the data name is not the same as coordinate, a coordinate with the same name will be searched for in the data object and the value_ranges will be used from it to set the intervals.

  • If not a dictionary and not None, it is interpreted as the

    interval description, the selection coordinate is taken the same as coordinate.

  • If None, the whole data interval will be used as a single

    interval.

optionsdict, optional, default=None

Options for APSD calculation. (Keys can be abbreviated.) Possible keys and values:

  • ‘Wavenumber’ (default=False):

    • bool: Whether to use 2*Pi*f for the output coordinate scale; this is useful for wavenumber calculation.

  • ‘Resolution’ (default=None):

    • Output resolution in the unit of the output coordinate.

  • ‘Range’ (default=None):

    • Output range in the unit of the output coordinate.

  • ‘Logarithmic’ (default=False):

    • bool: If True, will create logarithmic frequency binning.

  • ‘Interval_n’ (default=9):

    • int: Minimum number of intervals to use for the processing. These are identical length intervals inserted into the input interval list. Default is 8.

  • ‘Error calculation’ (default=True):

    • bool: Whether to calculate the error. Omitting error calculation increases speed. If ‘Interval_n’ is 1, no error calculation is done.

  • ‘Trend removal’ (default=[‘Poly’, 2]): Trend removal description. (See also flap.spectral_analysis._trend_removal().)

    • list:

      • ['poly', n]: Fit an n order polynomial to the data

        and subtract.

    • str:

      • ‘mean’: Subtract mean.

    • None: Don’t remove trend.

    Trend removal will be applied to each interval separately.

  • ‘Hanning’ (default=True):

    • bool: Whether to use a Hanning window.

Returns:
flap.DataObject
flap.data_object.ccf(object_name, ref=None, exp_id='*', output_name=None, coordinate=None, intervals=None, options=None)

N-dimensional cross-correlation function (CCF) or covariance calculation for the data object, taking ref as reference.

Calculate all CCF between all signals in ref and object_name, but not inside object_name and ref. Correlation is calculated along the coordinate(s) listed in coordinate, which should be identical for the to input data objects.

Returns a data object with dimension number object_name.dim+ref.dim-len(coordinate). The coordinates are replaced by coordinate_name+’ lag’. The CCF is calculated in multiple intervals (described by intervals) and the mean and variance will be returned.

Parameters:
object_namestr

Name identifying the data object in flap_storage.

refflap.DataObject, optional, default=None

Reference to use for CCF calculation. If None, object_name is used as reference.

exp_idstr, optional, default=’*’

Experiment ID. Supports extended Unix regular expressions.

output_namestr, optional, default=None

Name of the new data object added to flap_storage.

coordinatestr | list of str, optional, default=None

The name of the coordinate along which to calculate CCF, or a list of names. Each coordinate should change only along one data dimension and should be equidistant.

This and all other cordinates changing along the data dimension of these coordinates will be removed. New coordinates with coordinate_name+’ lag’ will be added.

intervalsdict | str, optional, default=None

Information of processing intervals.

  • If dictionary with a single key: {selection coordinate: description}). Key is a coordinate name which can be different from the calculation coordinate. Description can be Intervals, DataObject or a list of two numbers. If it is a data object with data name identical to the coordinate, the error ranges of the data object will be used for interval. If the data name is not the same as coordinate, a coordinate with the same name will be searched for in the data object and the value_ranges will be used from it to set the intervals.

  • If not a dictionary and not None, it is interpreted as the interval description, the selection coordinate is taken the same as coordinate.

  • If None, the whole data interval will be used as a single interval.

optionsdict, optional, default=None

Options for CCF calculation. (Keys can be abbreviated.) Possible keys and values:

  • ‘Resolution’ (default=None):

    • Output resolution for each coordinate. (Single value or list of values.)

  • ‘Range’ (default=None):

    • Output ranges for each coordinate. (List or list of lists.)

  • ‘Interval_n’ (default=8):

    • int: Minimum number of intervals to use for the processing. These are identical length intervals inserted into the input interval list. Default is 8.

  • ‘Error calculation’ (default=True):

    • bool: Whether to calculate the error. Omitting error calculation increases speed. If ‘Interval_n’ is 1, no error calculation is done.

  • ‘Trend removal’ (default=[‘Poly’, 2]): Trend removal description. (See also flap.spectral_analysis._trend_removal().)

    • list:

      • ['poly', n]: Fit an n order polynomial to the data and subtract.

    • str:

      • ‘mean’: Subtract mean.

    • None: Don’t remove trend.

    Trend removal will be applied to each interval separately.

  • ‘Normalize’ (default=False):

    • bool: whether to normalize with autocorrelations, that is, calculate correlation instead of covariance.

  • ‘Verbose’ (default=True):

    • bool: Whether to print progress messages.

Returns:
flap.DataObject
flap.data_object.cpsd(object_name, ref=None, exp_id='*', output_name=None, coordinate=None, intervals=None, options=None)

Complex cross-power spectral density (CPSD) calculation.

Calculate all spectra between all signals in ref and object_name, but not inside object_name and ref. Objects object_name and ref should both have the same equidistant coordinate with equal sampling points.

Returns a data object with dimension number object_name.dim+ref.dim-1. The coordinate is replaced by frequency or wavenumber. The spectrum is calculated in multiple intervals (described by intervals) and the mean and variance will be returned.

Parameters:
object_namestr

Name identifying the data object in flap_storage.

refflap.DataObject | str, optional, default=None

Reference to use for CPSD calculation, or name of reference in flap_storage. If None, object_name is used as reference.

exp_idstr, optional, default=’*’

Experiment ID. Supports extended Unix regular expressions.

output_namestr, optional, default=None

Name of the new data object added to flap_storage.

coordinatestr, optional, default=None

The name of the coordinate along which to calculate CPSD. This coordinate should change only along one data dimension and should be equidistant.

This and all other cordinates changing along the data dimension of this coordinate will be removed. A new coordinate with name ‘Frequency’/’Wavenumber’ will be added. The unit will be derived from the unit of the coordinate (e.g., Hz cm^-1, m^-1).

intervalsdict | str, optional, default=None

Information of processing intervals.

  • If dictionary with a single key: {selection coordinate: description}). Key is a coordinate name which can be different from the calculation coordinate. Description can be Intervals, DataObject or a list of two numbers. If it is a data object with data name identical to the coordinate, the error ranges of the data object will be used for interval. If the data name is not the same as coordinate, a coordinate with the same name will be searched for in the data object and the value_ranges will be used from it to set the intervals.

  • If not a dictionary and not None, it is interpreted as the interval description, the selection coordinate is taken the same as coordinate.

  • If None, the whole data interval will be used as a single interval.

optionsdict, optional, default=None

Options for APSD calculation. (Keys can be abbreviated.) Possible keys and values:

  • ‘Wavenumber’ (default=False):

    • bool: Whether to use 2*Pi*f for the output coordinate scale; this is useful for wavenumber calculation.

  • ‘Resolution’ (default=None):

    • Output resolution in the unit of the output coordinate.

  • ‘Range’:

    • Output range in the unit of the output coordinate.

  • ‘Logarithmic’ (default=False):

    • bool: If True, will create logarithmic frequency binning.

  • ‘Interval_n’ (default=8):

    • int: Minimum number of intervals to use for the processing. These are identical length intervals inserted into the input interval list. Default is 8.

  • ‘Error calculation’ (default=True):

    • bool: Whether to calculate the error. Omitting error calculation increases speed. If ‘Interval_n’ is 1, no error calculation is done.

  • ‘Trend removal’ (default=[‘Poly’, 2]): Trend removal description. (See also flap.spectral_analysis._trend_removal().)

    • list:

      • ['poly', n]: Fit an n order polynomial to the data

        and subtract.

    • str:

      • ‘mean’: Subtract mean.

    • None: Don’t remove trend.

    Trend removal will be applied to each interval separately.

  • ‘Hanning’ (default=True):

    • bool: Whether to use a Hanning window.

  • ‘Normalize’ (default=False):

    • bool: Whether to use normalization.

Returns:
flap.DataObject
flap.data_object.delete_data_object(object_name, exp_id='*')

Delete one or more data objects from flap_storage.

Parameters:
object_namestr | list of str

The name of one or more data objects to be deleted.

exp_idstr, optional, default=None

Experiment ID. Supports extended Unix regular expressions.

flap.data_object.detrend(object_name, exp_id='*', output_name=None, coordinate=None, intervals=None, options=None)

Trend removal.

Parameters:
object_namestr

Name identifying the data object in flap_storage.

exp_idstr, optional, default=’*’

Experiment ID. Supports extended Unix regular expressions.

output_namestr, optional, default=None

Name of the new data object added to flap_storage.

coordinatestr, optional, default=None

The ‘x’ coordinate for the trend removal.

intervalsdict | str, optional, default=None

Information of processing intervals.

  • If dictionary with a single key: {selection coordinate: description}). Key is a coordinate name which can be different from the calculation coordinate. Description can be Intervals, DataObject or a list of two numbers. If it is a data object with data name identical to the coordinate, the error ranges of the data object will be used for interval. If the data name is not the same as coordinate, a coordinate with the same name will be searched for in the data object and the value_ranges will be used from it to set the intervals.

  • If not a dictionary and not None, it is interpreted as the interval description, the selection coordinate is taken the same as coordinate.

  • If None, the whole data interval will be used as a single interval.

optionsdict, optional, default=None

Options for detrending. (Keys can be abbreviated.) Possible keys and values:

  • ‘Trend removal’: Trend removal description. (See also flap.spectral_analysis._trend_removal().)

    • list:

      • ['poly', n]: Fit an n order polynomial to the data and subtract.

    • str:

      • ‘mean’: Subtract mean.

    • None: Don’t remove trend.

Trend removal will be applied to each interval separately.

Returns:
flap.DataObject
flap.data_object.error_value(object_name, exp_id='*', output_name=None, options=None)

Return a DataObject containing the error of the data in the DataObject retrieved from flap_storage.

Parameters:
object_namestr

Name identifying the data object in flap_storage.

exp_idstr, optional, default=’*’

Experiment ID. Supports extended Unix regular expressions.

output_namestr, optional, default=None

Name of the new data object added to flap_storage.

optionsdict

Possible keys and values:

  • ‘High’:

    • bool: Use high error if error is asymmetric.

  • ‘Low’:

    • bool: Use low error is error is asymmetric

Returns:
flap.DataObject
flap.data_object.filter_data(object_name, exp_id='*', output_name=None, coordinate=None, intervals=None, options=None)

1D data filter.

Parameters:
object_namestr

Name identifying the data object in flap_storage.

exp_idstr, optional, default=’*’

Experiment ID. Supports extended Unix regular expressions.

output_namestr, optional, default=None

Name of the new data object added to flap_storage.

coordinatestr, optional, default=None

The ‘x’ coordinate for the filtering.

intervalsdict | str, optional, default=None

Information of processing intervals.

  • If dictionary with a single key: {selection coordinate: description}). Key is a coordinate name which can be different from the calculation coordinate. Description can be Intervals, DataObject or a list of two numbers. If it is a data object with data name identical to the coordinate, the error ranges of the data object will be used for interval. If the data name is not the same as coordinate, a coordinate with the same name will be searched for in the data object and the value_ranges will be used from it to set the intervals.

  • If not a dictionary and not None, it is interpreted as the interval description, the selection coordinate is taken the same as coordinate.

  • If None, the whole data interval will be used as a single interval.

optionsdict, optional, default=None

Options for filtering. Possible keys and values:

  • ‘Type’:

    • None: Do nothing.

    • ‘Int’: Single-term IIF filter, like RC integrator.

    • ‘Diff’: Single term IIF filter, like RC differentiator.

    • ‘Bandpass’ | ‘Lowpass’ | ‘Highpass’: Filters designed by scipy.signal.iirdesign. The filter type is in ‘Design’. Relation of filters and required cutting frequencies:

      • Bandpass: ‘f_low’ – ‘f_high’

      • Lowpass: – ‘f_high’

      • Highpass: – ‘f_low’

  • ‘Design’:

    • ‘Elliptic’ | ‘Butterworth’ | ‘Chebyshev I’ | ‘Chebyshev II’ | ‘Bessel’: The design type of the bandpass, lowpass or highpass filter.

      The numpy.iirdesign function is used for generating the filter.

      Setting inconsistent parameters can cause strange results. E.g. too high attenuation at too low frequency relative to the smapling frequency can be a problem.

  • ‘f_low’, ‘f_high’:

    • float: Cut on/off frequencies in Hz. (Middle between passband and stopband edge.)

  • ‘Steepness’:

    • float: Difference between passband and stopband edge frequencies as a fraction of the middle frequency.

  • ‘Loss’:

    • float: The maximum loss in the passband in dB.

  • ‘Attenuation’:

    • float: The minimum attenuation in the stopband dB.

  • ‘Tau’:

    • Time constant for integrator/differentiator (in units of the coordinate).

  • ‘Power’:

    • bool: Whether to calculate square of the signal after filtering.

  • ‘Inttime’:

    • Integration time after power calculation (in units of coordinate).

Returns:
flap.DataObject
flap.data_object.find_data_objects(name, exp_id='*')

Find data objects in flap_storage.

Parameters:
namestr

Name of data object.

exp_idstr, optional, default=’*’

Experiment ID. Supports extended Unix regular expressions.

Returns:
name_listlist

List of names of objects found.

exp_ID_listlist

List of experiment IDs of objects found.

flap.data_object.get_addcoord_function(data_source)

Return the add_coord function for a given data source.

Parameters:
data_sourcestr

The data source to use.

Returns:
callable
flap.data_object.get_data(data_source, exp_id=None, name=None, no_data=False, options=None, coordinates=None, object_name=None)

A general interface for reading data.

It will call the specific data read interface for the registered data sources.

Parameters:
data_sourcestr

The name of the data source.

exp_idstr, optional, default=None

Experiment ID.

namestr, optional, default=None

Name of data to retrieve.

no_databool, optional, default=False

Set to True to check data without reading it.

optionsobject, optional

Module-specific options.

coordinateslist of flap.Coordinate | dict, optional, default=None

Coordinates to use. Two possibilities:

1. List of Coordinate objects. These can precisely describe which part of the data to read.

  1. Dictionary. Each key is a coordinate name, the values can be:

    1. A list of two elements (describes a range in the coordinate).

    2. A single element. Will be converted into a list with two identical elements

    The dictionary will be converted to a list of Coordinate objects, which will be passed to the data source module.

object_namestr, optional, default=None

The name of the data object in flap_storage where data will be placed.

Returns:
flap.DataObject
flap.data_object.get_data_function(data_source)

Return the get_data function for a given data source.

Parameters:
data_sourcestr

The data source to use.

Returns:
callable
flap.data_object.get_data_object(object_name, exp_id='*')

Retrieve an object from flap_storage.

Note that the returned object is a copy, not a reference.

Parameters:
object_namestr

Name of the object to retrieve.

exp_idstr, optional, default=’*’

Experiment ID. Supports extended Unix regular expressions.

Returns:
flap.DataObject
flap.data_object.get_data_object_ref(object_name, exp_id='*')

Retrieve a reference to an object from flap_storage.

Note that the returned object is a reference, not a copy.

Parameters:
object_namestr

Name of the object to retrieve.

exp_idstr, optional, default=’*’

Experiment ID. Supports extended Unix regular expressions.

Returns:
flap.DataObject
flap.data_object.imag(object_name, exp_id='*', output_name=None)

Imaginary part of the data contained in the DataObject retrieved from flap_storage.

Has no effect on real-valued data.

Parameters:
object_namestr

Name identifying the data object in flap_storage.

exp_idstr, optional, default=’*’

Experiment ID. Supports extended Unix regular expressions.

output_namestr, optional, default=None

Name of the new data object added to flap_storage.

Returns:
flap.DataObject
flap.data_object.list_data_objects(name='*', exp_id='*', screen=True)

Prepare a printout of data objects.

Objects can either be from flap_storage or from listed data objects.

Parameters:
namestr | list of flap.DataObject

Name (with wildcards) or list of data objects.

exp_idstr

Experiment ID. Supports extended Unix regular expressions.

screenbool

Whether to print to screen.

Returns:
str
flap.data_object.list_data_sources()

Return a list of registered data source names as a list.

Returns:
list
flap.data_object.load(filename, options=None)

Load data saved by save().

If the data in the file were saved from flap_storage, it will also be loaded there, unless no_storage is set to True.

Parameters:
filenamestr

Name of the file to read.

optionsdict, optional, default=None

Possible keys and values:

  • ‘No storage’ (default=False):

    • bool: If True, don’t store data in flap_storage, just return a list of them.

Returns:
flap.DataObject | list of str

If data was not written from flap_storage, the original object will be returned. If it was written out from flap_storage, a list of the read objects will be returned.

flap.data_object.pdf(object_name, exp_id='*', coordinate=None, intervals=None, options=None, output_name=None)

Amplitude distribution function (PDF) of data.

Flattens the data array in the dimensions where the coordinates change and calculates PDF on this data for each case of the other dimensions.

Parameters:
object_namestr

Name identifying the data object in flap_storage.

exp_idstr, optional, default=’*’

Experiment ID. Supports extended Unix regular expressions.

coordinatestr | list of str, optional, default=None

The name of the coordinate(s) along which to calculate. If not set, the first coordinate in the data object will be used. These coordinates will be removed and replaced by a new coordinate with the name of the data.

intervalsdict | str, optional, default=None

Information of processing intervals.

  • If dictionary with a single key: {selection coordinate: description}). Key is a coordinate name which can be different from the calculation coordinate. Description can be Intervals, DataObject or a list of two numbers. If it is a data object with data name identical to the coordinate, the error ranges of the data object will be used for interval. If the data name is not the same as coordinate, a coordinate with the same name will be searched for in the data object and the value_ranges will be used from it to set the intervals.

  • If not a dictionary and not None, it is interpreted as the interval description, the selection coordinate is taken the same as coordinate (or its first element).

  • If None, the whole data interval will be used as a single interval.

optionsdict, optional, default=None

Options for generating the PDF. Possible keys and values:

  • ‘Range’:

    • list of float: The data value range. If not set, the data minimum-maximum will be used.

  • ‘Resolution’:

    • float: The resolution of the PDF.

  • ‘Number’:

    • The number of intervals in ‘Range’. This is an alternative to ‘Resolution’.

output_namestr, optional, default=None

Name of the new data object added to flap_storage.

Returns:
flap.DataObject
flap.data_object.phase(object_name, exp_id='*', output_name=None)

Phase of the data contained in the DataObject retrieved from flap_storage.

Parameters:
object_namestr

Name identifying the data object in flap_storage.

exp_idstr, optional, default=’*’

Experiment ID. Supports extended Unix regular expressions.

output_namestr, optional, default=None

Name of the new data object added to flap_storage.

Returns:
flap.DataObject
flap.data_object.plot(object_name, exp_id='*', axes=None, slicing=None, summing=None, options=None, plot_type=None, plot_options=None, plot_id=None, slicing_options=None)

Plotting for a data object in flap_storage.

Parameters:
object_namestr

Name of data object.

exp_idstr, optional, default=’*’

Experiment ID. Supports extended Unix regular expressions.

axeslist of str, optional, default=None

A list of coordinate names. They should be one of the coordinate names in the data object or ‘__Data__’. They describe the axes of the plot. If the number of axes is less than the number required for the plot, ‘__Data__’ will be added. If no axes are given, the default will be used, depending on the plot type. E.g. for an x-y plot the default first axis is the first coordinate, the second axis is ‘__Data__’.

slicingdict, optional, default=None

Passed as an argument to slice_data(). Slicing will be applied before plotting.

summingstr, optional, default=None

Passed as an argument to slice_data(). Slicing will be applied before plotting.

optionsdict, optional, default=None

Options for plotting. Possible keys and values:

  • ‘Error’ (default=True):

    • True: Plot all error bars

    • False: Do not plot errors.

    • int > 0: Plot this many error bars in plot.

  • ‘Y separation’ (default=None):

    • float: Vertical separation of curves in multi xy plot. For linear scale this will be added to consecutive curves. For Log scale, consecutive curves will be multiplied by this.

  • ‘Log x’ (default=False):

    • bool: Whether to use logscale X axis.

  • ‘Log y’ (default=False):

    • bool: Whether to use logscale Y axis.

  • ‘Log z’ (default=False):

    • bool: Whether to use logscale Z axis.

  • ‘All points’ (default=False):

    • bool: Default is False. If True, will plot all points, otherwise will plot only a reduced number of points (see ‘Maxpoints’). Each plotted point will be the mean of data in a box, and a vertical bar will indicate the value range in each box.

  • ‘Maxpoints’ (default=4000):

    • int: The maximum number of data points plotted. Above this, only this many points will be plotted if ‘All points’ is False.

  • ‘Complex mode’ (default=’Amp-phase’):

    • ‘Amp-phase’: Plot amplitude and phase.

    • ‘Real-imag’: Plot real and imaginary part.

  • ‘X range’,’Y range’ (default=None):

    • lists of two floats: Axes ranges.

  • ‘Z range’ (default=None):

    • list of two floats: Range of the vertical axis.

  • ‘Colormap’ (default=None):

    • str: Cmap name for image and contour plots.

  • ‘Levels’ (default=10):

    • int: Number of contour levels or array of levels.

  • ‘Aspect ratio’ (default=’auto’):

    • ‘equal’, ‘auto’ or float. (See imshow.)

  • ‘Waittime’ (default=1):

    • float: Time to wait in seconds between two images in anim-… type plots

  • ‘Video file’ (default=None):

    • str: Name of output video file for anim-… plots

  • ‘Video framerate’ (default=20):

    • float: Frame rate for output video.

  • ‘Video format’ (default=’avi’):

    • str: Format of the video. Valid options: ‘avi’.

  • ‘Clear’ (default=False):

    • bool: If True, don’t use the existing plots, generate new ones. (No overplotting.)

  • ‘Force axes’ (default=False):

    • True: Force overplotting, even if axes are incpomatible.

  • ‘Colorbar’ (default=True):

    • bool: Switch on/off colorbar

  • ‘Nan color’ (default=None):

    • The color to use in image data plotting for np.nan (not-a-number) values.

  • ‘Interpolation’ (default=’bilinear’):

    • Interpolation method for image plot.

  • ‘Language’ (default=’EN’):

    • str: Language of certain standard elements in the plot. Possible values: {‘EN’, ‘HU’}.

  • ‘EFIT options’ (default=None):

    • Dictionary of EFIT plotting options:

      • ‘Plot separatrix’: Set this to plot the separatrix onto the video.

      • ‘Separatrix X’: Name of the flap.DataObject for the separatrix X data (usually ‘R’).

      • ‘Separatrix Y’: Name of the flap.DataObject for the separatrix Y data (usually ‘z’).

      • ‘Separatrix XY’: Name of the 2D flap.DataObject for the separatrix XY data (usually ‘Rz’).

      • ‘Separatrix color’: Color of the separatrix for the plot.

      • ‘Plot limiter’: Set to plot the limiter onto the video.

      • ‘Limiter X’: Name of the flap.DataObject for the limiter X data (usually ‘R’).

      • ‘Limiter Y’: Name of the flap.DataObject for the limiter Y data (usually ‘z’).

      • ‘Limiter XY’: Name of the 2D flap.DataObject for the limiter XY data (usually ‘Rz’).

      • ‘Limiter color’: Color of the limiter for the plot.

      • ‘Plot flux surfaces’: Name of the 2D flap.DataObject for the flux surfaces (should have the same coordinate names as the plot).

      • ‘nlevels’: Number of contour lines for the flux surface plotting.

  • ‘Prevent saturation’ (default=False):

    • bool: Prevent saturation of the video signal when it exceeds zrange[1]. It uses data modulo zrange[1] ito overcome the saturation. (Works for animation.)

plot_type{‘xy’, ‘multi xy’, ‘grid xy’, ‘image’, ‘contour’, ‘anim-image’, ‘anim-contour’}, optional, default=None

The plot type. Can be abbreviated. Possible values:

  • ‘xy’: Simple 1D plot. Default axes are: first coordinate, ‘__Data__’.

  • ‘multi xy’: In case of 2D data, plots 1D curves with a vertical shift. Default x axis is the first coordinate, y axis is ‘Data’. The signals are labeled with the ‘Signal name’ coordinate, or with the one named in options['Signal name'].

  • ‘grid xy’: In case of 3D data, plots xy plots on a 2D grid. Axes are: grid x, grid y, x, y All coordinates should change in one dimension.

  • ‘image’: Plots a 2D data matrix as an image. Options: ‘Colormap’, ‘Data range’, …

  • ‘contour’: Contour plot.

  • ‘anim-image’, ‘anim-contour’: Like ‘image’ and ‘contour’, but the third axis is time.

plot_optionsdict | list of dict, optional, default=None

Dictionary containg matplotlib plot options. Will be passed over to the plot call. For plots with multiple subplots, this can be a list of dictionaries, each for one subplot.

plot_idflap.plot.PlotID, optional, default=None

Can be used for overplotting, if the new plot should go into an existing plot.

slicing_optionsdict, optional, default=None

Options for slicing. See slice_data() for details.

Returns:
flap.plot.PlotID

The plot ID of the current plot. This can be used later for overplotting.

flap.data_object.real(object_name, exp_id='*', output_name=None)

Real part of the data contained in the DataObject retrieved from flap_storage.

Has no effect on real-valued data.

Parameters:
object_namestr

Name identifying the data object in flap_storage.

exp_idstr, optional, default=’*’

Experiment ID. Supports extended Unix regular expressions.

output_namestr, optional, default=None

Name of the new data object added to flap_storage.

Returns:
flap.DataObject
flap.data_object.register_data_source(name, get_data_func=None, add_coord_func=None)

Register a new data source name and the associated functions.

Parameters:
namestr

The name of the data source.

get_data_funccallable, optional=None

The associated get_data function.

add_coord_funccallable, optional=None

The associated add_coord function.

Returns:
None | str
flap.data_object.remove_sharp_peaks(object_name, exp_id='*', coordinate=None, options=None, output_name=None)

Remove sharp peaks with only a few samples of duration from the data, e.g. due to neutron/gamma noise affecting APDCAM.

Peaks are detected by looking for sharp (as specified by the diff_limit option) rising and falling edges in the signal, within a window of maximum max_width_samples samples. Detected peaks can be removed from the signal by interpolation (see remove_peaks below).

By default, only a single data object is returned, with the number of removed/detected peaks and number of samples belonging to peaks added to the info field of the data object if return_num_detected_peaks is True. The keys added are: ‘num_peak_samples_detected’, ‘num_peaks_detected’, ‘num_peak_samples_removed’, ‘num_peaks_removed’.

If return_loc_detected_samples is set to True, an additional coordinate named ‘Peak detected’ is added to the data object, indicating whether a sample belongs to a detected peak.

If return_peak_shapes is set to True, the center times and the shapes (width, amplitude) of the detected peaks are also returned as two additional data objects (separately for each time series, if applicable). These data objects contain the center times of the peaks as a non-equidistant coordinate axis, with the width/amplitude of the peaks as the data.

Parameters:
object_namestr

Name identifying the data object in flap_storage.

exp_idstr, optional, default=’*’

Experiment ID. Supports extended Unix regular expressions.

coordinatestr, optional, default=None

The name of the coordinate along which to remove peaks. If not set, a coordinate named ‘Time’ in the DataObject will be used, if there is one. Must be equidistant and change along one dimension only. If the data changes along other dimensions as well, the peaks will be detected/removed separately for each slice along the other dimensions.

optionsdict, optional, default=None

Options for peak removal. Possible keys and values:

  • ‘max_width_samples’:

    • int: Maximum width of a peak in samples.

  • ‘diff_limit’:

    • The derivative limit for the detection of sharp rising/falling edges of peaks. Calculated in units of (signal units) / (time units).

  • ‘return_loc_detected_samples’ (default=False):

    • bool: Whether to return an additional coordinate axis kindicating if a sample belongs to a detected/removed peak.

  • ‘return_num_detected_peaks’ (default=False):

    • bool: Whether to return the number of detected peaks.

  • ‘return_peak_shapes’ (default=False):

    • bool: Whether to return the widths and amplitudes of detected peaks as two additional data objects.

  • ‘remove_peaks’ (default=True):

    • bool: Whether to remove the detected peaks.

  • ‘interpolation’ (default=’linear’):

    • ‘linear’ : Interpolation method for removing peaks.

output_namestr, optional, default=None

Name of the new data object added to flap_storage.

Returns:
flap.DataObject | (flap.DataObject, array of flap.DataObject, array of flap.DataObject)

If return_peak_shapes is False, a single data object is returned (with the peaks removed, if requested). If return_peak_shapes is True, a tuple of three data objects is returned: besides the one returned by default, the second containing the peak widths, and the third containing the peak amplitudes.

flap.data_object.save(data, filename, exp_id='*', options=None, protocol=3)

Save one or more data objects to a binary file.

Saving uses the pickle protocol.

Parameters:
dataflap.DataObject | str | list of str | object

Data to save, from various sources.

  • If flap.DataObject then save that object.

  • If str or list of str, then find these data objects in flap_storage and save them. Will also use exp_id to select data objects. These data objects can be restored into flap_storage using load.

  • If any other object, save it.

filenamestr

The name of the output file.

exp_idstr, optional, default=’*’

Experiment ID to use in conjuction with data.

optionsNone, optional, default=None

Currently unused.

protocolint

The pickle protocol to use.

flap.data_object.slice_data(object_name, exp_id='*', output_name=None, slicing=None, summing=None, options=None)

Slice a data object form flap_storage along one or more coordinates.

If output_name is set, the sliced object will be written back to flap_storage under this name.

Parameters:
object_namestr

Name of data object.

exp_idstr, optional, default=’*’

Experiment ID. Supports extended Unix regular expressions.

output_namestr, optional, default=None

The name of the new data object in the storage.

slicingdict, optional, default=None

Dictionary with keys referring to coordinates in the data object. Values can be:

  1. For SIMPLE SLICE: cases when closest value or interpolated value is selected.

    Possibilities:

    1. slice objects, range objects, scalars, lists, numpy array.

    2. DataObject objects without error and with data unit.name equal to the coordinate.

    3. DataObject with the name of one coordinate equal to the dictionary key without having value_ranges values.

    4. Intervals objects with one interval.

  2. For MULTI SLICE: Various range selection objects. In this case ranges are selected and a new dimension is added to the data array (only if more than 1 interval is selected) going through the intervals. If intervals are of different length, the longest will be used and missing elements filled with float('nan').

    Two new coordinates are added: “<coordinate> in interval”, “<coordinate> interval”.

    Possibilities:

    1. Intervals objects with more than one interval

    2. DataObject objects with data unit.name equal to the slicing coordinate. The error values give the intervals.

    3. DataObject with the name of one coordinate equal to the slicing coordinate. The value_ranges select the intervals.

    If range slicing is done with multiple coordinates which have common element in the dimension list, they will be done in one step. Otherwise the slicing is done sequentially.

summingdict, optional, default=None

Summing is applied to the sliced data. It processes data along one coordinate and the result is a scalar. This way summing reduces the number of dimensions.

Dictionary with keys referring to coordinates and values as processing strings. If the processed coordinate changes along multiple dimensions, those dimensions will be flattened.

For mean and avereage data, errors are calculated as error of independent variables. That is, taking the square root of the squared sum of errors. For coordinates the mean of the ranges is taken.

Processing strings are the following:

  • None: Nothing to be done in this dimension.

  • ‘Mean’: take mean of values in selection/coordinate.

  • ‘Sum’: take sum of values in selection/coordinate.

  • ‘Min’: take the minimum of the values/coordinate.

  • ‘Max’: take the maximum of the values/coordinate.

optionsdict, optional, default=True

Possible keys and values:

  • ‘Partial intervals’

    • bool: If True, processes intervals which extend over the coordinate limits. If False, only full intervals are processed.

  • ‘Slice type’:

    • ‘Simple’: Case a) above: closest or interpolated values are selected, dimensions are reduced or unchanged.

    • ‘Multi’: Case b) above: multiple intervals are selected, and their data is placed into a new dimension.

    • None: Automatically select. For slicing data in case b multi slice, otherwise simple

  • ‘Interpolation’:

    • ‘Closest value’

    • ‘Linear’

  • ‘Regenerate coordinates’: - bool: Default is True. If True, and summing is done, then looks for pairs of coordinates (‘Rel. <coord> in int(<coord1>)’, ‘Start <coord> in int(<coord1>)’).

    If such pairs are found, and they change on the same dimension or one of them is constant, then coordinate <coord> is regenerated and these are removed.

Returns:
flap.DataObject

The sliced object.

flap.data_object.stft(object_name, exp_id='*', output_name=None, coordinate=None, options=None)

Calculate the short-time Fourier transform (STFT) of the data in the DataObject retrieved from flap_storage, using scipy.signal.stft.

Parameters:
object_namestr

Name identifying the data object in flap_storage.

exp_idstr, optional, default=’*’

Experiment ID. Supports extended Unix regular expressions.

output_namestr, optional, default=None

Name of the new data object added to flap_storage.

coordinatestr, optional, default=None

The name of the coordinate along which to calculate the STFT. This coordinate should change only along one data dimension and should be equidistant.

This and all other coordinates changing along the data dimension of this coordinate will be removed. A new coordinate with name ‘Frequency’ (unit Hz) will be added.

optionsdict, optional, default=None

Options of STFT will be passed to scipy.signal.stft. Default options used are: {‘window’: ‘hann’, ‘nperseg’: 256, ‘noverlap’: None, ‘nfft’: None, ‘detrend’: False, ‘return_onesided’: True, ‘boundary’: ‘zeros’, ‘padded’: True}

Returns:
flap.DataObject