sporco.util

Utility functions

Functions

u(x) Python 2/3 compatible definition of utf8 literals
ntpl2array(ntpl) Convert a collections.namedtuple object to a numpy.ndarray object that can be saved using numpy.savez.
array2ntpl(arr) Convert a numpy.ndarray object constructed by ntpl2array back to the original collections.namedtuple representation.
transpose_ntpl_list(lst) Transpose a list of named tuple objects (of the same type) into a named tuple of lists.
tiledict(D[, sz]) Construct an image allowing visualization of dictionary content.
extractblocks(img, blksz[, stpsz]) Extract blocks from an ndarray signal into an ndarray.
averageblocks(blks, imgsz[, stpsz]) Average blocks together from an ndarray to reconstruct ndarray signal.
combineblocks(blks, imgsz[, stpsz, fn]) Combine blocks from an ndarray to reconstruct ndarray signal.
rgb2gray(rgb) Convert an RGB image (or images) to grayscale.
complex_randn(*args) Return a complex array of samples drawn from a standard normal distribution.
spnoise(s, frc[, smn, smx]) Return image with salt & pepper noise imposed on it.
rndmask(shp, frc[, dtype]) Return random mask image with values in \(\{0,1\}\).
pca(U[, centre]) Compute the PCA basis for columns of input array U.
tikhonov_filter(s, lmbda[, npd]) Lowpass filter based on Tikhonov regularization.
idle_cpu_count([mincpu]) Estimate number of idle CPUs, for use by multiprocessing code needing to determine how many processes can be run without excessive load.
grid_search(fn, grd[, fmin, nproc]) Perform a grid search for optimal parameters of a specified function.
convdicts() Access a set of example learned convolutional dictionaries.
netgetdata(url[, maxtry, timeout]) Get content of a file via a URL.
in_ipython() Determine whether code is running in an ipython shell.
in_notebook() Determine whether code is running in a Jupyter Notebook shell.
notebook_system_output() Get a context manager that attempts to use wurlitzer to capture system-level stdout/stderr within a Jupyter Notebook shell, without affecting normal operation when run as a Python script.

Classes

ExampleImages([scaled, dtype, zoom, gray, pth]) Access a set of example images.
Timer([labels, dfltlbl, alllbl]) Timer class supporting multiple independent labelled timers.
ContextTimer([timer, label, action]) A wrapper class for Timer that enables its use as a context manager.

Function Descriptions

sporco.util.u(x)[source]

Python 2/3 compatible definition of utf8 literals

sporco.util.ntpl2array(ntpl)[source]

Convert a collections.namedtuple object to a numpy.ndarray object that can be saved using numpy.savez.

Parameters:
ntpl : collections.namedtuple object

Named tuple object to be converted to ndarray

Returns:
arr : ndarray

Array representation of input named tuple

sporco.util.array2ntpl(arr)[source]

Convert a numpy.ndarray object constructed by ntpl2array back to the original collections.namedtuple representation.

Parameters:
arr : ndarray

Array representation of named tuple constructed by ntpl2array

Returns:
ntpl : collections.namedtuple object

Named tuple object with the same name and fields as the original named typle object provided to ntpl2array

sporco.util.transpose_ntpl_list(lst)[source]

Transpose a list of named tuple objects (of the same type) into a named tuple of lists.

Parameters:
lst : list of collections.namedtuple object

List of named tuple objects of the same type

Returns:
ntpl : collections.namedtuple object

Named tuple object with each entry consisting of a list of the corresponding fields of the named tuple objects in list lst

sporco.util.tiledict(D, sz=None)[source]

Construct an image allowing visualization of dictionary content.

Parameters:
D : array_like

Dictionary matrix/array.

sz : tuple

Size of each block in dictionary.

Returns:
im : ndarray

Image tiled with dictionary entries.

sporco.util.extractblocks(img, blksz, stpsz=None)[source]

Extract blocks from an ndarray signal into an ndarray.

Parameters:
img : ndarray or tuple of ndarrays

nd array of images, or tuple of images

blksz : tuple

tuple of block sizes, blocks are taken starting from the first index of img

stpsz : tuple, optional (default None, corresponds to steps of 1)

tuple of step sizes between neighboring blocks

Returns:
blks : ndarray

image blocks

sporco.util.averageblocks(blks, imgsz, stpsz=None)[source]

Average blocks together from an ndarray to reconstruct ndarray signal.

Parameters:
blks : ndarray

nd array of blocks of a signal

imgsz : tuple

tuple of the signal size

stpsz : tuple, optional (default None, corresponds to steps of 1)

tuple of step sizes between neighboring blocks

Returns:
imgs : ndarray

reconstructed signal, unknown pixels are returned as np.nan

sporco.util.combineblocks(blks, imgsz, stpsz=None, fn=<function median>)[source]

Combine blocks from an ndarray to reconstruct ndarray signal.

Parameters:
blks : ndarray

nd array of blocks of a signal

imgsz : tuple

tuple of the signal size

stpsz : tuple, optional (default None, corresponds to steps of 1)

tuple of step sizes between neighboring blocks

fn : function, optional (default np.median)

the function used to resolve multivalued cells

Returns:
imgs : ndarray

reconstructed signal, unknown pixels are returned as np.nan

sporco.util.rgb2gray(rgb)[source]

Convert an RGB image (or images) to grayscale.

Parameters:
rgb : ndarray

RGB image as Nr x Nc x 3 or Nr x Nc x 3 x K array

Returns:
gry : ndarray

Grayscale image as Nr x Nc or Nr x Nc x K array

sporco.util.complex_randn(*args)[source]

Return a complex array of samples drawn from a standard normal distribution.

Parameters:
d0, d1, …, dn : int

Dimensions of the random array

Returns:
a : ndarray

Random array of shape (d0, d1, …, dn)

sporco.util.spnoise(s, frc, smn=0.0, smx=1.0)[source]

Return image with salt & pepper noise imposed on it.

Parameters:
s : ndarray

Input image

frc : float

Desired fraction of pixels corrupted by noise

smn : float, optional (default 0.0)

Lower value for noise (pepper)

smx : float, optional (default 1.0)

Upper value for noise (salt)

Returns:
sn : ndarray

Noisy image

sporco.util.rndmask(shp, frc, dtype=None)[source]

Return random mask image with values in \(\{0,1\}\).

Parameters:
s : tuple

Mask array shape

frc : float

Desired fraction of zero pixels

dtype : data-type or None, optional (default None)

Data type of mask array

Returns:
msk : ndarray

Mask image

sporco.util.pca(U, centre=False)[source]

Compute the PCA basis for columns of input array U.

Parameters:
U : array_like

2D data array with rows corresponding to different variables and columns corresponding to different observations

center : bool, optional (default False)

Flag indicating whether to centre data

Returns:
B : ndarray

A 2D array representing the PCA basis; each column is a PCA component. B.T is the analysis transform into the PCA representation, and B is the corresponding synthesis transform

S : ndarray

The eigenvalues of the PCA components

C : ndarray or None

None if centering is disabled, otherwise the mean of the data matrix subtracted in performing the centering

sporco.util.tikhonov_filter(s, lmbda, npd=16)[source]

Lowpass filter based on Tikhonov regularization.

Lowpass filter image(s) and return low and high frequency components, consisting of the lowpass filtered image and its difference with the input image. The lowpass filter is equivalent to Tikhonov regularization with lmbda as the regularization parameter and a discrete gradient as the operator in the regularization term, i.e. the lowpass component is the solution to

\[\mathrm{argmin}_\mathbf{x} \; (1/2) \left\|\mathbf{x} - \mathbf{s} \right\|_2^2 + (\lambda / 2) \sum_i \| G_i \mathbf{x} \|_2^2 \;\;,\]

where \(\mathbf{s}\) is the input image, \(\lambda\) is the regularization parameter, and \(G_i\) is an operator that computes the discrete gradient along image axis \(i\). Once the lowpass component \(\mathbf{x}\) has been computed, the highpass component is just \(\mathbf{s} - \mathbf{x}\).

Parameters:
s : array_like

Input image or array of images.

lmbda : float

Regularization parameter controlling lowpass filtering.

npd : int, optional (default=16)

Number of samples to pad at image boundaries.

Returns:
sl : array_like

Lowpass image or array of images.

sh : array_like

Highpass image or array of images.

sporco.util.idle_cpu_count(mincpu=1)[source]

Estimate number of idle CPUs, for use by multiprocessing code needing to determine how many processes can be run without excessive load. This function uses os.getloadavg which is only available under a Unix OS.

Parameters:
mincpu : int

Minimum number of CPUs to report, independent of actual estimate

Returns:
idle : int

Estimate of number of idle CPUs

Perform a grid search for optimal parameters of a specified function. In the simplest case the function returns a float value, and a single optimum value and corresponding parameter values are identified. If the function returns a tuple of values, each of these is taken to define a separate function on the search grid, with optimum function values and corresponding parameter values being identified for each of them. On all platforms except Windows (where mp.Pool usage has some limitations), the computation of the function at the grid points is computed in parallel.

Warning: This function will hang if fn makes use of pyfftw with multi-threading enabled (the bug has been reported). When using the FFT functions in sporco.linalg, multi-threading can be disabled by including the following code:

import sporco.linalg
sporco.linalg.pyfftw_threads = 1
Parameters:
fn : function

Function to be evaluated. It should take a tuple of parameter values as an argument, and return a float value or a tuple of float values.

grd : tuple of array_like

A tuple providing an array of sample points for each axis of the grid on which the search is to be performed.

fmin : bool, optional (default True)

Determine whether optimal function values are selected as minima or maxima. If fmin is True then minima are selected.

nproc : int or None, optional (default None)

Number of processes to run in parallel. If None, the number of CPUs of the system is used.

Returns:
sprm : ndarray

Optimal parameter values on each axis. If fn is multi-valued, sprm is a matrix with rows corresponding to parameter values and columns corresponding to function values.

sfvl : float or ndarray

Optimum function value or values

fvmx : ndarray

Function value(s) on search grid

sidx : tuple of int or tuple of ndarray

Indices of optimal values on parameter grid

sporco.util.convdicts()[source]

Access a set of example learned convolutional dictionaries.

Returns:
cdd : dict

A dict associating description strings with dictionaries represented as ndarrays

Examples

Print the dict keys to obtain the identifiers of the available dictionaries

>>> from sporco import util
>>> cd = util.convdicts()
>>> print(cd.keys())
['G:12x12x72', 'G:8x8x16,12x12x32,16x16x48', ...]

Select a specific example dictionary using the corresponding identifier

>>> D = cd['G:8x8x96']
sporco.util.netgetdata(url, maxtry=3, timeout=10)[source]

Get content of a file via a URL.

Parameters:
url : string

URL of the file to be downloaded

maxtry : int, optional (default 3)

Maximum number of download retries

timeout : int, optional (default 10)

Timeout in seconds for blocking operations

Returns:
str : io.BytesIO

Buffered I/O stream

Raises:
urlerror.URLError (urllib2.URLError in Python 2,
urllib.error.URLError in Python 3)

If the file cannot be downloaded

sporco.util.in_ipython()[source]

Determine whether code is running in an ipython shell.

Returns:
ip : bool

True if running in an ipython shell, False otherwise

sporco.util.in_notebook()[source]

Determine whether code is running in a Jupyter Notebook shell.

Returns:
ip : bool

True if running in a notebook shell, False otherwise

sporco.util.notebook_system_output()[source]

Get a context manager that attempts to use wurlitzer to capture system-level stdout/stderr within a Jupyter Notebook shell, without affecting normal operation when run as a Python script. For example:

>>> sys_pipes = sporco.util.notebook_system_output()
>>> with sys_pipes():
>>>    command_producing_system_level_output()
Returns:
sys_pipes : context manager

Context manager that handles output redirection when run within a Jupyter Notebook shell


Class Descriptions

class sporco.util.ExampleImages(scaled=False, dtype=None, zoom=None, gray=False, pth=None)[source]

Bases: object

Access a set of example images.

Parameters:
scaled : bool, optional (default False)

Flag indicating whether images should be on the range [0,…,255] with np.uint8 dtype (False), or on the range [0,…,1] with np.float32 dtype (True)

dtype : data-type or None, optional (default None)

Desired data type of images. If scaled is True and dtype is an integer type, the output data type is np.float32

zoom : float or None, optional (default None)

Optional support rescaling factor to apply to the images

gray : bool, optional (default False)

Flag indicating whether RGB images should be converted to grayscale

pth : string or None (default None)

Path to directory containing image files. If the value is None the path points to a set of example images that are included with the package.

images()[source]

Get list of available images.

Returns:
nlst : list

A list of names of available images

groups()[source]

Get list of available image groups.

Returns:
grp : list

A list of names of available image groups

groupimages(grp)[source]

Get list of available images in specified group.

Parameters:
grp : str

Name of image group

Returns:
nlst : list

A list of names of available images in the specified group

image(fname, group=None, scaled=None, dtype=None, idxexp=None, zoom=None, gray=None)[source]

Get named image.

Parameters:
fname : string

Filename of image

group : string or None, optional (default None)

Name of image group

scaled : bool or None, optional (default None)

Flag indicating whether images should be on the range [0,…,255] with np.uint8 dtype (False), or on the range [0,…,1] with np.float32 dtype (True). If the value is None, scaling behaviour is determined by the scaling parameter passed to the object initializer, otherwise that selection is overridden.

dtype : data-type or None, optional (default None)

Desired data type of images. If scaled is True and dtype is an integer type, the output data type is np.float32. If the value is None, the data type is determined by the dtype parameter passed to the object initializer, otherwise that selection is overridden.

idxexp : index expression or None, optional (default None)

An index expression selecting, for example, a cropped region of the requested image. This selection is applied before any zoom rescaling so the expression does not need to be modified when the zoom factor is changed.

zoom : float or None, optional (default None)

Optional rescaling factor to apply to the images. If the value is None, support rescaling behaviour is determined by the zoom parameter passed to the object initializer, otherwise that selection is overridden.

gray : bool or None, optional (default None)

Flag indicating whether RGB images should be converted to grayscale. If the value is None, behaviour is determined by the gray parameter passed to the object initializer.

Returns:
img : ndarray

Image array

Raises:
IOError

If the image is not accessible

class sporco.util.Timer(labels=None, dfltlbl='main', alllbl='all')[source]

Bases: object

Timer class supporting multiple independent labelled timers.

The timer is based on the relative time returned by timeit.default_timer.

Parameters:
labels : string or list, optional (default None)

Specify the label(s) of the timer(s) to be initialised to zero.

dfltlbl : string, optional (default ‘main’)

Set the default timer label to be used when methods are called without specifying a label

alllbl : string, optional (default ‘all’)

Set the label string that will be used to denote all timer labels

start(labels=None)[source]

Start specified timer(s).

Parameters:
labels : string or list, optional (default None)

Specify the label(s) of the timer(s) to be started. If it is None, start the default timer with label specified by the dfltlbl parameter of __init__.

stop(labels=None)[source]

Stop specified timer(s).

Parameters:
labels : string or list, optional (default None)

Specify the label(s) of the timer(s) to be stopped. If it is None, stop the default timer with label specified by the dfltlbl parameter of __init__. If it is equal to the string specified by the alllbl parameter of __init__, stop all timers.

reset(labels=None)[source]

Reset specified timer(s).

Parameters:
labels : string or list, optional (default None)

Specify the label(s) of the timer(s) to be stopped. If it is None, stop the default timer with label specified by the dfltlbl parameter of __init__. If it is equal to the string specified by the alllbl parameter of __init__, stop all timers.

elapsed(label=None, total=True)[source]

Get elapsed time since timer start.

Parameters:
label : string, optional (default None)

Specify the label of the timer for which the elapsed time is required. If it is None, the default timer with label specified by the dfltlbl parameter of __init__ is selected.

total : bool, optional (default True)

If True return the total elapsed time since the first call of start for the selected timer, otherwise return the elapsed time since the most recent call of start for which there has not been a corresponding call to stop.

Returns:
dlt : float

Elapsed time

labels()[source]

Get a list of timer labels.

Returns:
lbl : list

List of timer labels

class sporco.util.ContextTimer(timer=None, label=None, action='StartStop')[source]

Bases: object

A wrapper class for Timer that enables its use as a context manager.

For example, instead of

>>> t = Timer()
>>> t.start()
>>> do_something()
>>> t.stop()
>>> elapsed = t.elapsed()

one can use

>>> t = Timer()
>>> with ContextTimer(t):
...   do_something()
>>> elapsed = t.elapsed()
Parameters:
timer : class:Timer object, optional (default None)

Specify the timer object to be used as a context manager. If None, a new class:Timer object is constructed.

label : string, optional (default None)

Specify the label of the timer to be used. If it is None, start the default timer.

action : string, optional (default ‘StartStop’)

Specify actions to be taken on context entry and exit. If the value is ‘StartStop’, start the timer on entry and stop on exit; if it is ‘StopStart’, stop the timer on entry and start it on exit.

elapsed(total=True)[source]

Return the elapsed time for the timer.

Parameters:
total : bool, optional (default True)

If True return the total elapsed time since the first call of start for the selected timer, otherwise return the elapsed time since the most recent call of start for which there has not been a corresponding call to stop.

Returns:
dlt : float

Elapsed time