DefocusTracker: A modular toolbox for defocusing-based, single-camera, 3D particle tracking

The need for single-camera 3D particle tracking methods is growing, among others, due to the increasing focus in biomedical research often relying on single-plane microscopy imaging. Defocusing-based methods are ideal for a wide-spread use as they rely on basic microscopy imaging rather than requiring additional non-standard optics. However, a wide-spread use has been limited by the lack of accessible and easy-to-use software. DefocusTracker is an open-source toolbox based on the universal principles of General Defocusing Particle Tracking (GDPT) relying solely on a reference look-up table and image recognition to connect a particle's image and its respective out-of-plane depth coordinate. The toolbox is built in a modular fashion, allowing for easy addition of new image recognition methods, while maintaining the same workflow and external user interface. DefocusTracker is implemented in MATLAB, while a parallel implementation in Python is in the preparation.


Introduction
The use of single-camera 3D particle tracking analysis is receiving increasing interest, among others, due to the rapid development of bio-engineering and biomedical sciences where single-access imaging, such as with microscopes, is a standard research tool [15,16]. For this, methods based on the principle of particle image defocusing are particularly attractive as no special optics or cameras are required, and have potential for wide-spread use. However, until now most of the software for defocused-based particle tracking has been developed in-house for private use of research groups, and there are only few examples of user-friendly software that can be accessible to a larger audience, including researchers outside the engineering or computer-science community. One example is GDPTlab, a MATLAB GUI implementation written by the authors and released in 2015 [5]. GDPTlab was used by few research groups and cited in several peer-reviewed journals (see also Reuse potential section). GDPTlab, however, was not distributed under an open-source license, thus its potential for collaboration and expansion was limited by that.
To accommodate this need, we developed DefocusTracker, which is a modular and open-source toolbox for defocusing-based 3D particle tracking. DefocusTracker uses different architecture and functions and is not compatible with GDPTlab, however they are based on the same method, namely the General Defocusing Particle Tracking (GDPT). GDPT relies on a reference look-up table, with known defocused particle images and depth positions, and on a image recognition method, that matches target and reference particle images [1,3,15]. Following machine learning terminology, we will refer here to the look-up table as the training set, whereas the image recognition method will be part of the chosen calibration model. The GDPT principle is shown in Fig. 1. Panel (a) shows the creation and training of a calibration model through calibration/training images of defocused particle images with known 3D particle positions. In Panel (b) the trained model is used to reconstruct the 3D particle positions from the particles' defocused images in 2D measurement images. For more details on the GDPT method, experiments, and uncertainty assessment, we refer to Refs. 1, 3, 5.
DefocusTracker is built in a modular fashion to allow for a continuous addition of state-of-the-art image recognition methods, e.g. methods based on convolutional neural networks and deep learning, while maintaining general method-independent features. The current implemented method, referred to as Method 1, is based on the normalized cross-correlation function and described in Ref. 10. DefocusTracker is implemented in MATLAB, while a parallel implementation in Python is in the preparation. Defocus-Tracker is accompanied by a website, https://defocustracking.com/, to facilitate the research community with a platform for sharing of user guides, experiences, and applications, as well as for data for training and validation.
Implementation and architecture General architecture  The toolbox uses three types of data structures (rectangles) and five main functions (round shapes, green). The toolbox is modular, allowing for addition and use of different models for the image processing and particle tracking.
The general architecture and workflow is shown in Fig. 1(c). The toolbox is based on three types of data structures and five primary functions for their creation, processing, and manipulation: Data structures: imageset Link and description to a set of images, i.e. image paths, number of images, and image type.
dataset Particle data, i.e. 3D spatial positions and displacements, trajectories, connection to image frames, and detection accuracy estimation.
model Data and settings required to process the images, i.e. method-specific parameters as well as training and processing settings.

Functions:
create() Creates a data structure.
show() Opens GUIs to inspect data structures.
train() Trains a model on a specific training set (imageset+dataset). process() Processes an imageset using a given model. postprocess() Manipulates a dataset, e.g. merge datasets, apply scaling, remove outliers, perform particle tracking, filter trajectories, and estimate uncertainties.
Workflow: As seen in Fig. 1(c), a typical workflow starts with the creation of a model using the create() function. Each model refers to a specific method (e.g. Method 1) and it is at first created with some default values. The model is trained using the train() function by feeding a training set as input. A training set consists of a dataset and an imageset (made with create()), corresponding to a set of images containing one or more particles of known 3D position. As illustrated in Fig. 1(a), such a training set can be obtained experimentally by taking subsequent images of particles displaced at known positions, e.g. by observing particles sedimented on a microchannel bottom, while taking images at known objective distances using a focusing stage. If only a subset of the calibration particles are used for a training, the remaining particles can be used as validation to make a pre-measurement uncertainty estimation.
With a trained model as input, the process() function can take one or more measurement images in an imageset and output a dataset containing the measured 3D particle positions. The dataset can be further manipulated via the postprocess() function for purposes such as outlier removal or trajectory smoothing. Throughout the entire workflow, the show() function can be used to visualize and inspect the data structures.

Modularity and Method 1:
The DefocusTracker toolbox is modular in the sense that a model can be created based on different methods for the image recognition. If a new method is added to the toolbox, the data structures, primary functions, and workflow will remain the same. The current implementation provides only one method for the creation of a model, namely Method 1. For a full description of Method 1, we refer to [10]. Briefly, Method 1 is based on training images of a single particle (the so-called calibration stack) and uses the normalized cross-correlation function for image recognition. The normalized cross-correlation is used to rate the similarity between a target particle image and the calibration stack images, using its maximum peak value as the similarity coefficient, referred to as C m . The values of C m can range from 0 to 1, with 1 corresponding to a perfect match between the target image and a calibration image.

MATLAB implementation
DefocusTracker is implemented in MATLAB and additionally requires the image processing, curve fitting, and statistics toolboxes. The implementation is script-based with certain features using GUI-based pop-up windows for visualization. The data structures are so-called MATLAB structs, namely structure arrays where data is grouped using containers called fields. The data in a field is accessed using dot notation of the form structName.fieldName, e.g. the path of an imageset is called with imageset.path. The primary toolbox functions follow the form of standard MATLAB functions and are named as dtracker functionName(), e.g. dtracker create(). A full overview of the data structures and functions are shown in Fig. 2. An exemplary practical application of DefocusTracker, including a script and few results, is presented in Fig. 3. More details about this application are given in the Quality control section.

Python implementation
A Python implementation of DefocusTracker is planned and under development and it will the lines of the MATLAB implementation. The data structures will be implemented using Python dictionaries, whereas the functions will be part of the module dtracker. Following the above example, the path of an imageset will be called in Python with imageset['path'], whereas the create() function will be called as dtracker.create().
The Python implementation is available on the Gitlab repository: https://gitlab. com/defocustracking/defocustracker-python. Updates and release information can be followed on https://defocustracking.com.  Figure 3: Example workflow of the toolbox MATLAB implementation. The example workflow is based on part of the provided Work-Through Example 2 that takes the user through the processing and analysis of particle trajectories inside an evaporating droplet [12]. (a) Illustration of the code used to create and train a model (green frame), to process image test data (blue frame), and to process measurement images (red frame). (b) Illustration of the implemented pop-up GUIs used to visualize and inspect the data structures.

Quality control
The MATLAB toolbox has been tested functionally on Windows 10 with MATLAB releases R2018b and 2020a. The toolbox performance has been tested and investigated extensively in two recent publications [3,10]. In Barnkob and Rossi [3], guidelines for assessing the uncertainty of GDPT analyses were given. Synthetic images were used to test the toolbox in terms of measurement uncertainty and relative number of measured particles as a function of image signal-to-noise ratio, particle image concentration, and variations in image intensity. In Rossi and Barnkob [10], different toolbox settings were tested on synthetic and experimental images to outline the measurement uncertainties, detection rates, and processing times. The results were benchmarked against the GDPTlab software [5], which has been extensively-tested and used in high-impact research publications, see more in the Section Reuse potential.
The MATLAB toolbox contains two work-through examples (WTE1 and WTE2) that serve as tutorials to get new users quickly started, but also as test scripts in case new functionalities or methods are added. The scripts of each WTE are included in the DefocusTracker package, while the relative datasets can be acquired via https: //defocustracking.com/datasets/. WTE1 is based on synthetic images and gives a first introduction to the basic building blocks of the toolbox. The use of synthetic images allow for an exact estimation of the uncertainty using the postprocessing method 'compare true values'. WTE2 is based on a state-of-the art microfluidic experiment, namely the 3D flow inside an evaporating droplet [12], and guides the user toward a more advanced use of DefocusTracker, including postprocessing and bias correction. We report in Fig. 3 a shortened version of the WTE2 script, including few screenshots of GUI panels obtained with the dtracker show() function. For the full commented version we refer to the script Work through ex2.m. As the community grows, we expect that more WTEs will be added by the users and developers.
(2) Availability Operating system Windows, UNIX/Linux, Macintosh (and any operating system supporting MATLAB).

Dependencies
The MATLAB implementation requires the additional MATLAB toolboxes: 'curve fitting toolbox', 'image toolbox', 'statistics toolbox' (3) Reuse potential The analysis of particle positions, velocities, and trajectories is an integral part of many research disciplines. This includes analyses in 2D as well as in 3D, and with the rapid growth in fields relying on microscopy, such single-camera methods can provide unique and important information. One example is within the field of microfluidics where high control of fluid flow and externally-applied forces is becoming an important tool in biomedical research and applications. Here, 3D detection and tracking of particles and cells can provide the necessary information needed for optimization, standardization, and real-time inspection and control [13,14]. GDPT has shown to be an excellent candidate for a wide-spread technique as is a simple and universal defocusing-based method and requires no special optics and can be used in standard microscope setups. Here, the development of free, accessible, user-friendly, and accurate tools can greatly enhance the practicability and availability of the method. One example is the MATLAB implementation GDPTlab (also by the authors), which has been distributed to researchers since 2015 and has proven its value in a number of research projects including work in journals such as Proceedings of the National Academy of Sciences, Physical Review Letters, and Scientific Reports [2, 4, 6-9, 11, 16-18]. Note that most of this research were done in laboratories with no previous experience of 3D single-camera particle tracking prior to the use of GDPTlab. GDPTlab has thus shown the huge potential such methods hold, if free and user-friendly implementations are available. Though GDPTlab has enabled many researchers to get started using GDPT, it has unfortunately not been fully accessible as an open-source project, limiting its further development and adaptation by the community. Defocus-Tracker fills this need as it is fully open-source. The toolbox contains a readme-file and work-through examples for users to get quickly started.

Modification and support
DefocusTracker is set up in a versatile and modular fashion allowing for easy expansions and improvements, such as extensions of custom functionalities and features, e.g. using MATLAB's GUI editor and pre-built functions. In the Python implementation, such expansions could involve the use of popular libraries for data analysis and machine learning, such as SciKit, Keras, and TensorFlow. DefocusTracker is supported by https://defocustracking.com/ which is an online platform created to assist the development and support of DefocusTracker as well as to facilitate the research community with a place for sharing of data and experiences related to single-camera 3D particle tracking. The platform contains several forums, e.g. for new developers to request access and for users to ask the community for support.

Funding statement
The research leading to these results has received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie