BayesFit is a module for Python that allows users to fit models to psychophysical data using Bayesian inference. The module aims to make it easier to develop probabilistic models for psychophysical data in Python by providing users with a simple API that streamlines the process of defining psychophysical models, obtaining fits, extracting outputs, and visualizing fitted models. Our software implementation uses numerical integration as the primary tool to fit models, which avoids the complications that arise in using Markov Chain Monte Carlo (MCMC) methods [

Fitting statistical models to behavioural data is an important part of analyzing task performance within the field of Psychology [

Although fitting a psychometric function to data is a common component of the analysis of behavioural data, very few tools have been designed to help researchers streamline this fitting process [

The full API documentation for BayesFit can be found at:

To demonstrate the functionality of this software, below we provide the results from a simulated two-interval, forced-choice (2-IFC) detection experiment in which each experimental trial consists of two successive, temporal intervals, and the observer must determine whether the target stimulus was presented in the first or second interval. Stimulus intensity is varied across trials, and response accuracy varies from near chance levels at low intensity to nearly perfect accuracy at high stimulus intensity (see Figure

Plot of synthetic data generated from an observer detecting the flash of a light onscreen. The abscissa represents stimulus intensity, while the ordinate axis represents proportion correct.

BayesFit expects that data provided by the user are organized into a

Example of how data from a single psychophysical experiment should be formatted when passing these data to BayesFit for model fitting.

Once data are appropriately organized, we are ready to use BayesFit to perform the model fitting procedure. We first import the BayesFit module into our workspace via:

Next, we specify the output variables and options that we want to use for our fitting procedure. BayesFit uses only one main function called

Although we have listed all options that are available to the user in fitting models using BayesFit, the only mandatory argument to provide the

Using these outputted variables, we can generate a plot of the fitted model using the

As can be seen in the example above, only a few lines of code are needed to generate parameter estimates for our model and quickly visualize the results (see Figure

Plot of psychometric function using a Cumulative Normal sigmoid fit to data using BayesFit.

Our example above used an MLE fitting procedure, as no prior distributions were provided to BayesFit for parameters to-be-estimated. However, if we have prior knowledge about what the values of our parameters should be, along with what the shape of the distribution in likelihood of different values are (e.g., Uniform, Normal, etc.), we can pass these arguments along to the

BayesFit provides users with 5 different distributions that can be used to specify priors for each parameter. These include:

Therefore, if the user knew that certain values were more likely compared to others for the parameters controlling the scale and slope of the psychometric function, then the user could pass those arguments along to the

From this example, it can also be seen that choosing prior distributions, and the values that define them, can be difficult. Therefore, it is best to use priors only when you are confident that the probability of certain values for parameters to-be-estimated are more likely than others based on theoretical or empirical grounds. Following the fitting procedure, the user can now visualize the priors used during the fitting procedure using the

Plot of prior distributions used during Bayesian inference of parameters of psychometric function.

The marginal distributions for each parameter can be plotted via the

Plot of marginal distributions of parameters extracted from posterior.

The posterior surface can be visualized via the

Plot of posterior surface for scale and slope parameters, collapsed across gamma and lambda.

Occasions might arise where the user would like to automatically, rather than manually, fit multiple psychometric models across several datasets. The BayesFit module makes this task easy by requiring only two small changes to the procedure used to fit a single model. The first change is that the batch input argument for the

Example of how multiple datasets should be combined into a single dictionary object before being passed as an argument to BayesFit for batch fitting.

Here is a simple example of using the

Note that only user-specified values for fixed parameters are used when performing batch fitting. Furthermore, to prevent memory issues, only select metrics will be saved for each psychometric model fit.

In order to assess the accuracy of parameter estimates using BayesFit, we generated 100 synthetic datasets from simulated observers in a psychophysical experiment. The datasets were generated to mimic the response of observers performing a 2-interval forced choice task with with eight equally-spaced stimulus intensities (range: 0.1–0.9). The true probability for an observer of correctly responding to a given level of intensity,

where

These parameters were used to generate 100 binomial random responses at each stimulus intensity, for a total of 800 responses. In other words, 100 responses were simulated for each observer at each stimulus intensity. Data from each simulated observer were fit using the

Results regarding the accuracy of parameter estimates for α, β, and λ are shown in Figures

Three plots comparing the accuracy of predicted parameter values versus the true values used to generate data for each simulated observer. The ordinate axis provides the value of the parameter, either estimated or true, and the abscissa is an index for each simulated observer.

Boxplots of parameter estimates generated using BayesFit compared to target distributions.

These simulations demonstrate that BayesFit can be trusted for use in estimating the parameters of models used to fit psychophsyical data. It should be noted that, when possible, greater care should be taken in choosing the options used to fit models to psychophysical data. However, these simulations serve as a worst case scenario where the user fails to specify prior knowledge that may help the fitting procedure, and yet, BayesFit still performs extremely well.

To evaluate the performance of BayesFit in comparison to Psignifit 4.0, the only other module the authors are aware of for fitting models to psychophysical data in Python (albeit 2.x), we make use of the simulated data used above for judging the accuracy of BayesFit in parameter estimation. Because BayesFit uses definitions for psychometric functions according to those specified in [

However, we can assess the accuracy of threshold estimates obtained from the model fit to a simulated observer’s dataset using each module. Therefore, we used Psignifit 4.0 to fit models to simulated data using default settings except for specifying the sigmoid type as a Weibull function, and also defining threshold at 75% proportion correct response.

Figure

Boxplots of distributions for values of threshold estimated at 75% correct performance using BayesFit and Psignifit 4.0 compared to target distribution.

BayesFit was written in Python [

Schematic of the architecture for BayesFit, also displaying module dependencies for each function, whether directly or using a function that also depended upon use of a specific module.

Both unit and functional testing has been performed on BayesFit. Travis CI (

Although BayesFit undergoes extensive testing to ensure the software is functioning properly, if any issues arise during use, please let us know under the issue tracker for BayesFit on Github:

BayesFit should function on any software capable of running Python 3.5 or greater. Current operating systems that BayesFit has been tested on include macOS Sierra, Windows 10, and Ubuntu 16.04.

Python (version 3.5 or greater.)

The minimum system requirements needed to run Python 3.5 or greater.

numpy >= 0.19.0

matplotlib >= 2.0.0

scipy >= 1.1.0

Michael Slugocki is the sole developer, and current maintainer of the BayesFit module. The code for this software has been developed in the lab of Dr. Allison B. Sekuler and Dr. Patrick J. Bennett, who are co-authors of this article.

English

The need to estimate parameters of models fit to psychophysical data exists in many different domains within Psychology. Therefore, the potential for reuse of this software to be high. The core code of BayesFit has been written using only the Numpy module, and therefore many functions in BayesFit can be used in other projects that want to use numerical integration and Bayesian inference in modelling psychophysical data. Some highlighted functionality includes:

Checking user arguments for fitting psychometric functions.

Extracting user provided prior distributions.

Defining psychometric function definitions.

Computing likelihood surface via numerical integration.

Computing posterior distribution via numerical integration.

Compute various parameters from fitted psychometric function (e.g., MAP estimate, Bayesian Credible Intervals, etc).

Plotting model fits from parameter estimates.

Plotting prior, marginal, and posterior distributions.

We thank the editorial team and reviewers for their insightful comments and efforts spent towards improving this manuscript, and the BayesFit module.

The authors have no competing interests to declare.