Fan-Slicer: A Pycuda Package for Fast Reslicing of Ultrasound Shaped Planes

Fan-Slicer (https://github.com/UCL/fan-slicer) is a Python package that enables the fast sampling (slicing) of 2D ultrasound-shaped images from a 3D volume. To increase sampling speed, CUDA kernel functions are used in conjunction with the Pycuda package. The main features include functions to generate images from both 3D surface models and 3D volumes. Additionally, the package also allows for the sampling of images from curvilinear (fan shaped planes) and linear (rectangle shaped planes) ultrasound transducers. Potential uses of Fan-slicer include the generation of large datasets of 2D images from 3D volumes and the simulation of intra-operative data among others


(1) OVERVIEW INTRODUCTION
Fan-slicer is a package designed for the sampling/ simulation of ultrasound shaped planes from a preoperative scan such as Computed Tomography (CT) or Magnetic Resonance Imaging (MRI). This software has been initially implemented as part of an imaging pipeline to aid the development of ultrasound guidance algorithms for laparoscopic liver surgery [1,2] and endoscopic interventions [3]. Given a set of Laparoscopic Ultrasound (LUS) images and a pre-operative 3D scan, the resampling of LUS planes in pre-operative space enables both the implementation of image registration pipelines and visualisation of the corresponding results. MATLAB 1 and 3D Slicer 2 have functionalities to perform this sampling operation. However, none of these software tools allow for the fast simulation of smaller planes bounded by the ultrasound fan shape taken at an arbitrary position and orientation in 3D space. Since speed and easy integration of this sampling is a key requirement for registration pipelines in medical imaging applications, we have designed a parallel solution in CUDA that allows for the sampling of multiple planes. Initially implemented in MATLAB with the Parallel Computing Toolbox and CUDA kernels written in C++, the software was later implemented in Python and Pycuda for easier deployment. Currently, this software has enabled research on new registration methods of LUS to CT scans of the liver using Contentbased Image Retrieval [1,2,4,5] and the training of General Adversarial Networks (GANs) for the simulation of Ultrasound images from abdominal CT [6].

IMPLEMENTATION AND ARCHITECTURE
Fan-slicer is implemented with Python and CUDA kernels written in C++ that are compiled using Pycuda. A simple overview of the package functionality is described in Figure 1. The project structure is generated from the PythonTemplate of Scikit-Surgery [7]. The package consists of two main classes for the sampling of 2D images -IntensityVolume in intensity_volume.py samples 3D volumes (intensity) and SegmentedVolume in segmented_volume.py samples 3D surfaces (binary). Their implementation is briefly described in Figure 2.

Upon instantiation, both SegmentedVolume and
IntensityVolume allocate volume data in a 3D array. IntensityVolume creates a single array from either a NumPy array, a NifTI file (.nii) or a DICOM file.
SegmentedVolume receives a variable number of surfaces in VTK format, performs voxelisation, and then outputs a separate binary 3D array for each of them. As an intermediate step, VTK files are converted to simpler mesh structures described in mesh.py -code can be adapted to other formats as long as this mesh structure is obtained.
To minimise data transfer between CPU and GPU and therefore increase sampling speed, all volume and image data is pre-allocated to the GPU upon instantiation. Besides the 3D data, both classes receive as input a configuration file (.json) with the ultrasound image shape parameterisation and the number of images (an integer) that should be simulated per run. The configuration can be either linear or curvilinear and has variable parameters described in USING.rst. By knowing the configuration and image number to be simulated, the classes pre-allocate a set of fixed size GPU arrays for the slicing task.

IMAGE SAMPLING
Slicing of 2D images is achieved with the function simulate_image for both classes by providing the number of images to slice and an array with a corresponding number Figure 1 Overview of Fan-slicer package pipelines. Given a virtual ultrasound transducer pose, the package can generate ultrasound shaped images from segmented medical surfaces by voxelising binary volumes or directly from 3D volumetric medical images. of concatenated 4 × 4 poses composed of rotation and translation. If the number of images is different from the one used in the constructor, the software repeats the preallocation step. To generate the input number of images, simulate_image calls a method that uses a sequence of CUDA kernels loaded from cuda_reslicing.py (see Figure 2). Depending on the class and image parameterisation used, a specific combination of kernels listed in Table 1. is used.
For all slicing options, the first kernel computes point clouds from poses (kernels highlighted with (1)). Then, depending on the class (binary or intensity), an interpolation kernel is used (kernels highlighted with (2)). If the configuration is curvilinear, a third kernel must be used to warp the interpolated result into a 2D fan-shaped grid (kernels highlighted with (3)). If the configuration is linear there is no need for a third kernel as the interpolation result is already in linear coordinates.

QUALITY CONTROL
Unit tests in tests/test_pycuda_simulations.py are used to test the image simulation with both intensity and binary models, both with linear and curvilinear shapes.
These tests have been checked in Windows and Linux environments. In addition to the tests, simulation_demo.py provides a simple demo on how to simulate images using both intensity and binary models. Therefore, to check if the package is working, a user should: • Run the unit tests in tests/test_pycuda_simulations.py. • Run the script simulation_demo.py and check if the plotted image results are the same as the ones stored in the demo_outputs folder.

OPERATING SYSTEM
Minimum versions tested: