SnappySonic: An Ultrasound Acquisition Replay Simulator

SnappySonic provides an ultrasound acquisition replay simulator designed for public engagement and training. It provides a simple interface to allow users to experience ultrasound acquisition without the need for specialist hardware or acoustically compatible phantoms. The software is implemented in Python, built on top of a set of open source Python modules targeted at surgical innovation. The library has high potential for reuse, most obviously for those who want to simulate ultrasound acquisition, but it could also be used as a user interface for displaying high dimensional images or video data.


Introduction
The use of ultrasound acquisition simulators for medical training purposes is increasing and has been shown to provide benefit for training purposes [27,28]. Removing the need for actual ultrasound saves equipment costs, and allows for more tightly controlled training environments. Currently available ultrasound simulators range from combinations of specialized hardware [6, 14,15] that provide a tactilely realistic experience to phone based apps [1,13] that provide convenient training examples. Serious Games for ultrasound training have also been developed using simulated ultrasound images [24] and more abstract representations [20]. However, the available platforms are limited in their configuration options, being tailored to training clinicians on specific anatomical examples. Lack of configuration makes it difficult if you wanted to use your own recorded ultrasound data or configure the geometry to your hardware requirements.
SnappySonic was developed to form part of a public engagement exercise targeted at school age children and their parents/carers. The aim was to demonstrate to the general public (families and our local community) some of the challenges in interpreting ultrasound images. We did this via a serious game, where the user was required to identify objects in recorded ultrasound images. The software provides a semi-realistic experience of ultrasound acquisition, whilst avoiding the need for novice users to cope with such issues as maintaining acoustic contact and beam angle. The software reads a video buffer of appropriate pre-recorded images, and looks up and displays an appropriate image based on the position of a tracked object under the user's control. Tracking can be done either with a webcam and ArUco markers [25] or with one of NDI's [7] tracking systems. The user can configure the images to be shown and the geometry of the tracking system. The video buffer we used, together with an example configuration file can be downloaded from the source code repository. The software has been tested by approximately 100 users at a public engagement event, during which we evaluated its performance with a questionnaire, see section 3.1.

Implementation and architecture
SnappySonic was developed in Python using dependencies available from the Python Packaging Index [12]. Three dependencies are from the SNAPPY [17] software libraries under development by the authors to support innovation in surgical and interventional sciences. There are three further direct dependences on PySide2, NumPy [32] and OpenCV [23]. Figure 1 shows the dependency graph for SnappySonic.
SnappySonic implements an OverlayApp class which inherits from scikit-surgeryutils OverlayBaseApp. OverlayBaseApp implements a Qt widget capable of showing a video image overlaid with a VTK [30] renderer. SnappySonic implements the update member function to take input from a physical tracking system, using scikit-surgerynditracker, or scikit-surgeryarucotracker. The tracker position is used to select an image from an image buffer which is preloaded when the class is initialised. The image buffer used and how the images are selected can be controlled via a configuration file written in JavaScript Object Notation [5]. An image buffer containing ultrasound images of four household items together with an example configuration file can be downloaded from the source repository. Figure 2 shows a screen shot of the software in use. The interface uses two separate windows, one showing the tracking information, so the user knows where they are in the tracked volume, while the second shows the pseudo ultrasound image. It is possible to place the windows on separate screens to prevent the user seeing the tracking information. Figure 3 shows the system in use, coupled with a plastic torso phantom. We attached an ArUco tag to an obsolete ultrasound probe for a more realistic experience, however it is not necessary to use an ultrasound probe.
robust, reusable libraries to support translational research in surgery, [29]. Well defined software process [31] is central to the development process. WEISS operates its own quality management system (QMS), implementing the IEC EN 62304:2006 standard "Medical device software -Software life cycle processes" [26] to enable the deployment of software to theatre. The software described in this paper falls outside the QMS as it is not a medical device, however we take care to follow as much of the standard as practical, to allow the component software to be used in a medical device at some future date.
We use our own GitLab [3] server [21] for project management, and implement GitLab-CI [4] for continuous integration testing. We use the issue tracker functionality of GitLab to document bug reports and feature requests and reference the issues from code commits. This creates a link between software requirements and development steps, in line with Section 6 of [26].
We use a test driven development process [22]. New features or bug fixes are first defined via a set of failing unit tests. Code changes are then made to get the unit tests passing. GitLab-CI is used to monitor the status of these and existing tests to ensure that any changes do not cause regression of any existing requirements. Documentation is generated from the source tree using sphinx [16]. The status of unit tests, coverage, and documentation is communicated to users via flags on the project home page and on the project's PyPi page.

Functional Testing
The performance of the software was assessed during a public engagement held at WEISS. The ultrasound replay simulator was set up similarly to Figure 3 and members of the public were asked to work out what what objects were "hidden" in boxes, based on interaction with pre-recorded ultrasound images of the objects. A form (Figure 4) was used to determine whether the users had been able to interpret the images correctly. Qualitatively, users agreed that the simulation gave a good experience of ultrasound scanning. Quantitative results are in Table 1.

Availability and Support
SnappySonic can be installed on supported platforms using the pip [9] installation tool, or downloaded from the source repository [18].
We welcome feature requests and bug reports, which can be submitted via the source repository's [18] issue tracker, or by emailing the lead author.

Operating system
SnappySonic is available for Python 3.6 onwards and has been tested on Linux, Windows, and MacOS.

Programming language
Python 3.6

Additional system requirements
A webcam is required to use the ArUco based tracking system.

List of contributors
The contributors are the listed authors.

Reuse potential
SnappySonic can most obviously be reused by anyone who wants to create a customisable ultrasound acquisition simulator, either using the ultrasound data buffer within the source repository or by recording their own data. However reuse is not limited to ultrasound data. The interface could be used to navigate quickly through any image buffer so may have Thompson  applications in video navigation or navigation though multidimensional medical image data sets.
The dependent libraries can also be reused individually or in combination. The software may be forked via gitlab which implements issue trackers to enable bug reporting and feature requests.

Funding statement
This work is supported by the Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS) (203145Z/16/Z).  A screen shot of the system in use, top left is the command line and console output. Bottom left a window showing the tracker position with respect to the different parts of the video buffer. At right is the recorded ultrasound image. In this example the image is of a latex glove filled with water, which the user is trying identify by moving the probe around in "Box A". Thompson   The software in use during our "Science of Surgery" event. We attached a printed ArUco tag to an obsolete ultrasound probe to provide a sense of realism. The user moves the probe over a plastic torso phantom, the probe is tracked by a webcam on top of the monitor, and the ultrasound image shown changes depending on where the probe is over the phantom. Images by James Tye. Thompson   We evaluated the functional performance of the ultrasound simulator during a public engagement event. We asked participants to use the system to identify what household object was "in the box", from a selection of eight possible objects shown on this form. Thompson  The results of the functional test. 35 users filled in the form. Most users were able to correctly identify objects based on the recorded ultrasound. The orange was notably more challenging to identify.