Introduction

In the data science age, the ability to effectively explore and understand data is a central task. In service of this task, unsupervised machine learning provides many useful tools for exploring and understanding data in a mostly assumption-free manner [11]. One of the most widely used approaches for doing so is clustering, which offer researchers sophisticated algorithms for grouping observations to maximize similarity in feature space, and in so doing, uncover natural, non-random structure in data [1]. Among the most commonly used techniques are k-means and Gaussian mixture models for clustering.

K-means is a clustering technique that groups observations in a shared space based on spatial similarity, in pursuit of intra-cluster homogeneity and inter-cluster heterogeneity [2]. The goal is to minimize within cluster sums of squares based on distances to a computed cluster centroid, which in the k-means case, is a computed cluster mean.¹

The goal of using Gaussian mixture models for clustering applications is similar to the k-means goal of maximizing similarity across observations in a shared space. Yet the key difference is that cluster assignment in the mixture model case is determined probabilistically [3]. Observations that have similar probabilities of belonging to the same component (or “cluster”) are grouped together, but with a degree of uncertainty. A further difference is the presence and use of probabilities, making most mixture models parametric. As such, the Expectation-Maximization (EM) algorithm is an iterative algorithm that is most often used to estimate the parameters, which constrain the clustering procedure [10]. Once each of the components in the mixed space are identified, clustering is carried out by assigning each observation to the most likely component in the mixture, which is determined by maximizing the posterior probabilities of each observation belonging to each component.

In this paper, we present two original Shiny apps written in the R language [4], that allow researchers in any field to explore, understand, and directly compare clustering results from these widely used clustering algorithms. The first Shiny app is based on simulated data to ease the user into the logic of clustering, and the second leverages the Iris data, and is expanded to also include observation-level numeric output (e.g., cluster assignments) in addition visual output. Both apps leverage ggplot2 graphics [5], which contribute to clean, simple user interfaces to contribute to deeper understanding.

While our main goal is to encourage exposure to and interaction with these widely used clustering algorithms, our Shiny apps can easily be extended in future projects by, for example, including other data sets as well as other algorithms to widen to comparability. Indeed, as the apps are developed openly, we encourage such contributions. Avenues for contribution are discussed below as well as at the corresponding Github repository under the “Contribution” section in the README.

Implementation and architecture

The software for each app was implemented by programming a user interface (UI) and a server, as necessary component parts of a Shiny app. Upon creation of the UI and server, each app was consolidated into a single script, and then deployed at the corresponding author’s Shiny host site, pdwaggoner.shinyapps.io. More details on deployment are available in the following “Availability” section.

To get a clearer sense of the architecture of each app, consider the screenshots of the launched apps below in Figures 1, 2, 3. Figure 1 shows the first clustering app using simulated data. Figure 2 shows the visual component of the second clustering app using the Iris data. Figure 3 shows the numeric output from the second clustering app also using the Iris data.

In Figure 1, the raw data densities are displayed along with an optional histogram for another view, as well as the computed cluster densities. The numeric output just below the visual output compares across each technique as well as the original raw inputs. P1 and P2 correspond with the input features, mu1 and mu2 correspond with the density means, and sigma1 and sigma2 correspond with the variance of each component/cluster. Users are simply able to slide the nodes in the left-hand panel to see the densities and clusters change in real time. Users also have a number of display options in the lower right, including the probability density functions (components), as well as a rug plot of the observations.

Further, in the first app in Figure 1, data are drawn from a mixture of two univariate Gaussian distributions. The first distribution has mean 0 and standard deviation 1 while the mean mu2, the standard deviation sigma2 and the proportion p2 of the second distribution are defined by the user. The sample size can also be altered, ranging from 50 to 2000.

As users are able to determine the aspects included on the visual rendering of the patterns (e.g., including a scatter plot of the data, a histogram, the probability distribution density of each component), the objective of this app is to show that k-means clustering is well adapted when the two distributions are quite similar, except for a difference in the mean. On the other hand, Gaussian mixtures are better suited when the distributions have quite different standard deviations and/or proportions.

Turning to the second app in Figure 2, we use the famous (Fisher’s or Anderson’s) Iris data set, which gives the measurements in centimetres of the variables sepal length and width and petal length and width, respectively, for 50 flowers from each of three species of iris: Iris setosa, versicolor, and virginica.

In the second app, users can select the variables to display on the X and Y axes, as well as the ellipses sizes, which is a 2D extension of a prediction interval for bivariate distributions. It is important to note that a mixture of multivariate Gaussian distributions that takes into account correlations between variables results in an excellent classification (only 2% of misclassified flowers). See http://sia.webpopix.org/mixtureModels.html for more details about the algorithms used in these apps.

Quality control

The apps have been thoroughly quality checked and tested locally. First, the UI’s have been tested on the several widely used internet platforms including Firefox Quantum 65.0.2, Chrome 72.0.3626.121, Safari 12.0.3, and Opera 58.0.3135.107. No errors or problems occurred on any browser or system. The apps worked efficiently and as expected on all platforms.

Second, the code was locally tested using the shinytest package, which is a unit testing package for Shiny apps. This package tests for the following, which was copied and pasted from RStudio for consistency:²

1. “First is the test driver. This is the R process that coordinates the testing and controls the web browser. When working on creating tests interactively, this is the R process that you use.
2. Next is the Shiny process, also known as the server. This is the R process that runs the target Shiny application.
3. Finally, there is the web browser, also known as the client, which connects to the server. This is a headless web browser – one which renders the web page internally, but doesn’t display the content to the screen (PhantomJS).”

Across all of these local tests for both apps, which are catalogued at the Github repository under the “tests” folder for the simulated data app (1) and “tests_iris” for the Iris data app (2), there were zero errors thrown, and all code and apps behaved as expected.

Finally, at the Github repository listed below, users can inspect all source code, unit testing results, as well as inspect the README for more details on each app as well as how to diagnose the results both visually and numerically (see the “Understanding the Output” subsection in the README).

Operating system

The apps were originally built and launched on MacOS Mojave 10.14.3.

Programming language

The apps were originally written in R v3.5.2 and RStudio v1.2.1206.

Dependencies

shiny [4]; ggplot2 [5]; htmltools [6]; ellipse [7]; shinyWidgets [8]; kmeans (Base R); mixtools [9].

List of contributors

• – Marc Lavielle, Center of Applied Mathematics, Inria & Ecole Polytechnique
• – Philip D. Waggoner, Computational Social Science, University of Chicago

Software location

Language

English