pyparty: Blob Detection, Drawing and Manipulation in Python

Introduction Open-source image processing tools are rapidly spreading across programming languages and scientific domains. The widely-used ImageJ[1] platform, along with its distributions such as Fiji[2], offer an excellent Java-based suite. In the last few years, image processing and computer vision in Python has rapidly matured1 with the emergence of libraries like scikit-image[3], Mahotas[4], and SimpleCV[5]. scikit-image and Mahotas utilize 2d NumPy[6] arrays for their image data structures, thus providing a familiar API to users of NumPy and the other core SciPy libraries, especially Matplotlib[7], IPython[8]. The motivation for pyparty arose from difficulties encountered when quantifying nanoparticle distributions on silica substrates. We had found that after acquiring an image, preprocessing, segmentation and labelling techniques were easily applied; however, separating and measuring the various particle species in post-processing was difficult. This is primarily because the objects of interest were stored as boolean or integer-labeled arrays, which become unwieldy under heavy manipulation. pyparty emerged as a means to abstract the concept of particles (i.e. image blobs) into custom data structures for intuitive manipulation and characterization whilst preserving the image API. In addition to integrating new particle constructs with the existing array and image processing functions, pyparty extends scikit-image’s rasterization toolbox with new particle types, patterning, and an interface to Matplotlib patch objects for vectorized particle renderings. Thus, pyparty leverages the conventional imaging pipeline at both ends; it provides a tool set for artificial image composition, and a framework for particle post-processing.


Introduction
Open-source image processing tools are rapidly spreading across programming languages and scientific domains. The widely-used ImageJ [1] platform, along with its distributions such as Fiji [2], offer an excellent Java-based suite. In the last few years, image processing and computer vision in Python has rapidly matured 1 with the emergence of libraries like scikit-image [3], Mahotas [4], and SimpleCV [5]. scikit-image and Mahotas utilize 2d NumPy [6] arrays for their image data structures, thus providing a familiar API to users of NumPy and the other core SciPy libraries, especially Matplotlib [7], IPython [8].
The motivation for pyparty arose from difficulties encountered when quantifying nanoparticle distributions on silica substrates. We had found that after acquiring an image, preprocessing, segmentation and labelling techniques were easily applied; however, separating and measuring the various particle species in post-processing was difficult. This is primarily because the objects of interest were stored as boolean or integer-labeled arrays, which become unwieldy under heavy manipulation.
pyparty emerged as a means to abstract the concept of particles (i.e. image blobs) into custom data structures for intuitive manipulation and characterization whilst preserving the image API. In addition to integrating new particle constructs with the existing array and image processing functions, pyparty extends scikit-image's rasterization toolbox with new particle types, patterning, and an interface to Matplotlib patch objects for vectorized particle renderings. Thus, pyparty leverages the conventional imaging pipeline at both ends; it provides a tool set for artificial image composition, and a framework for particle post-processing.

Implementation and architecture
Rasterization, as well as particle labels and descriptors (e.g. area, eccentricity etc...), are all deferred to algorithms implemented in scikit-image [14][15][16]. pyparty adds a convenience layer to extend some of this functionality. For example, pyparty's implements a Grid object for patterning, and provides a simple framework for drawing multiparticles, for example circle dimers and trimers.
The Canvas is pyparty's primary datastructure. The Canvas is comprised of a background image, a Grid and ParticleManager, which is a container for particle storage and manipulation. The Canvas fully decouples particle information such as position, color, and pixel from the background image. Since particle information is stored in python containers, slicing, arithmetic, boolean indexing and other common operations are very easy to perform. For example, an operation like dilate all circular particles in an image that are over a certain size and mean brightness is a simple task in pyparty. In addition to the Canvas, the MultiCanvas is provided for images with multiple particle types.
Finally, pyparty simplifies some common image processing tasks involving thresholding, artificial noise generation, color assignment, image type conversions, plotting and blob filtering, albeit not to the extent of IJBlob [9].

Examples of Use
We will first highlight pyparty's drawing capabilities to create pseudo electron microscope images. A more involved version is used to compare segmentation algorithms in a SOftwarE MEtapapEr pyparty: Blob Detection, Drawing and Manipulation in Python corollary study [10]. In the example code snippets, Canvas and MultiCanvas methods are bolded for clarity. To begin, let us define the parameters corresponding to ranges for particle size and intensity, the noise fill-fraction and the image resolution. For brevity, we load a previously generated pyparty background image (see documentation) of a trigonometrically-varying contrast gradient. The Canvas crops the image to our desired resolution upon initialization. Next, randomly-sized ellipses are added, and the Canvas grid is set to 20 pixels per division; this controls the inter-particle spacing. This results in 380 particles covering 23% of the image surface as shown in Figure 1. Next, we apply a Gaussian filter through scikit-image (σ = 3px) to smooth the particle boundaries. This negligibly affects the background, which varies over a much larger scale. In pyparty, spectral noise can be assigned according to any NumPy distribution function (Gaussian, gamma etc..), but for clearer visualization, we'll use black pixels, also known as "pepper" noise (see Figure 2). from skimage.filter import gaussian_filter from pyparty.noise import pepper from pyparty.plots import zoom smooth = gaussian_filter(cnvs.image, 3) noisy = pepper(smooth, NOISE) zoom(noisy, (0, 0, 250, 250)) As a final exercise, we employ the MultiCanvas to characterize nanoparticle species in an electron microscope image. The image, which is packaged with pyparty, has been segmented using Ilastik's [11] object classification workflow into groups labelled as singles, dimers, trimers and clusters as shown in Figure 3. pyparty has an API for shape filtering, but it is not explored here. The MultiCanvas   is built directly from the image via the from_labels() constructor. Persistent names and colors are assigned to the labels for easier identification between plots. Figures 4 and Figure 5 shows pyparty's decomposition of the image into these colored particle groups.

Quality control
Development and testing is performed on Mac, Linux and Windows 7 systems. Tutorials and case studies are provided as IPython notebooks, which are also integrated into a quasi regression test suite.
(2) availability (3) reuse potential Although developed in the context of nanotechnology, pyparty's post-processing and drawing capabilities are general to 2D imaging workflows, especially those found in microscopy 2 . pyparty is currently used for in-house research pursuits [13] and continues to develop. New applications like Ilastik integrate semi-supervised learning and object classification tools with unprecedented accessibility. This trend suggests that the need for general particle analysis tools will increase in coming years. As image analysis in Python continually expands to meet the needs of researchers, we believe pyparty will emerge as a small but important component in particle analysis workflows.

acknowledgements
Thank you Annie Matsko for excellent revision suggestions. I'd also like to thank the scikit-image team for several helpful discussions that ultimately inspired this project, and for their exhaustive efforts in creating a superb library.

Notes
1 SciPy's ndimage collection pre-dates scikit-image for example. 2 Cell profiler [12] is a great example of a specialized microscopy application in Python.