Introduction

Design of experiments (DOE) includes various statistical approaches for planning, executing and analysing experiments in a systematic and efficient way. Definitive Screening Design (DSD) [1] is a particularly useful DOE technique in fields such as material engineering [2, 3, 4], chemistry [5, 6] and biology [7, 8], where numerous factors are involved in the experiments. The DSD has significant advantages over conventional DOEs. Most conventional DOE techniques are for experiments with two levels (e.g. –1/+1) for all factors. If there is a second-order effect of factor A (i.e. AA) that deals with two levels, a conventional DOE cannot find an optimal condition that may exist between the two levels. Even though some conventional DOEs can deal with three levels (e.g., –1/0/+1), such as L18 orthogonal array, the second-order effects often involve correlations with other factor effects. This implies that ambiguity may remain in the process of building models. In contrast, DSD deals with three levels (low, middle, and high values) of the factors (parameters) and the correlations between the factors are minimized. These features are useful for building second-order models and finding the optimal levels of factors in the investigated parameter windows of the experiments.

However, designing and analyzing DSD experiments typically requires commercial software (e.g., JMP, Design-Expert and Minitab) or programming knowledge (e.g., R or Python), which is, in many cases, a barrier for researchers to employ DSD in their experiments. Considering this situation, the authors of this work developed DSDApp, a free web application that provides an effortless way to use DSD for researchers with a lack of access to commercial software packages or statistical programming knowledge.

The user-friendly interface of DSDApp allows users to design DSD experiments, to create second-order models (including main effects, two-factor interactions and quadratic effects) and to perform parameter optimization to obtain desirable objective values. The app can be used in a very simple way only by button-clicking, which makes DSDApp a more attractive and easier option to employ DSD than existing R packages (e.g., “daewr”) or Python libraries (e.g., “definitive-screening-design”) that require statistical programming experience.

DSDApp is particularly profitable for researchers in small research or academic organizations or companies without access to commercial software. Also, those without any previous experience with DOE techniques or with limited experience in the use of DSD may benefit from the app because of its simpler and more intuitive operation compared to commercial software that offer complicated functionalities for more specialized needs. Overall, DSDApp is expected to extend the use of DSD and broaden experiments in various research and development fields.

Implementation and architecture

As shown in Figure 1, DSDApp offers three main functionalities: planning definitive screening designs, making regression models, and optimizing parameters. The detailed usage of DSDApp is explained in the following sections.

Planning DSD

In the “Plan” tab, one can create a DSD table with 4 to 12 factors. The process should start by changing the levels (low: –1, middle:0, high:1) of the factors. These levels are corresponding to the minimum, middle, and maximum values of the factors. Figure 2 shows an example of a six-factor (A, B, C, D, E, F) DSD table generated by DSDApp. Each row is an experimental run that users should execute with the factor levels given, and collect result Y. According to Ref. 9, two or more “fake factors” should be considered when constructing a DSD table for better effect detection performance. Fake factors do not correspond to the actual variables (therefore no need to consider when doing experiments), but only act when building a model as described in Appendix.

The table generated can be downloaded as a csv file by clicking on “Download.” In the experiment, the actual data should be recorded and added in a new column in the downloaded csv file.

In Figure 2, instead of actual data, the checkbox “Generate the sample data” was ticked to generate sample data given by:

This model will be used for testing the app in the section “Quality control”.

Making Models

Before making the models, it is necessary to upload the file with the DSD table and experimental result(s). Multiple objective variables can be included in the added columns. By clicking on “Browse” (Figure 3), it is possible to upload a csv/txt file in the DSDApp from your computer.

The models can be built by following the steps below.

1. Set the result column as an objective variable “Y”, input variables (A, B, C, D) as “X,” and fake factors (E, F) as “Fake Factor.”
2. Click on “Find active terms” to obtain active main factors and second-order factors (Step 1). The detection of main and second-order factors is described in the tab “Step 1” as shown in Figure 4.
3. After pressing “Find active terms,” the possible first- and second-order terms appear in “X1” and “X2” tab automatically, following the model building strategy detailed in the Appendix. The user can consider which terms should be included in the model and manually add/remove factors of interest (Step 2).
4. (Optional) If the user manually altered terms in “X1”, press “Regenerate X2” to regenerate terms in “X2” by combining the first-order terms selected in “X1.” This procedure is optional if the user is satisfied with the terms that automatically appeared in “X1” and “X2” after pressing “Find active terms”.
5. Click on “Build model” to make a model with the terms “X1” and “X2.” The summary of the model is described in the tab “Step2” as shown in Figure 5.

DSDApp employs a model-selection strategy tailored for DSD, where active main (first-order) factors are detected first and then second-order terms related to main factors are included [9]. For example, when main factors A and B are active (or set as X1), possible second-order terms (X2) are AA, AB and BB. To avoid overfitting, the second-order terms are included to minimize Akaike Information Criteria with finite correction (AICc). See the Appendix for a more detailed explanation.

After creating the model, the prediction of the output value is possible in the tab “Predict”. The input vector × (or the factor levels) can be set to specified values, as it can be seen in Figure 6. The prediction value y_x0 at x₀ = [1, A, B, C, D, …] and its prediction interval is calculated as:

where X is the design matrix of DSD, α is the significance level (0.05), n is the number of runs, p is the number of terms in the model (including the intercept term) and $t_{\alpha /2,n-p}$M8 \documentclass[10pt]{article} \usepackage{wasysym} \usepackage[substack]{amsmath} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage[mathscr]{eucal} \usepackage{mathrsfs} \usepackage{pmc} \usepackage[Euler]{upgreek} \pagestyle{empty} \oddsidemargin -1.0in \begin{document} \[{t_{\alpha /2,n - p}}\] \end{document} is t-value with two-sided confidential level α and the degree of freedom n-p.

Optimizing Factor Level

DSDApp offers the possibility to optimize the parameters, i.e., to find the optimal set of parameters for the main factors to obtain the desired objective variable(s). DSDApp transforms the objective value y into a “desirability function” that evaluates how satisfactory y is on a scale from 0 “not satisfactory” to 1 “completely satisfactory.” Figure 7 illustrates the desirability functions for three different cases: minimization, maximization and tuning y at a specific value. For minimizing and maximizing, the individual desirability function D₁ is expressed as:

where y_allowable refers to the minimum required value that the user wishes to secure, and y_target refers to the ideal value. In the case of minimization, for example, if y > y_allowable, D₁ gets smaller than 0.01 as shown in Figure 7(a). This value of D₁ is small enough to consider y as unsatisfactory when y > y_allowable and, therefore, y tends to be smaller than y_allowable. In the case of maximization, the value of D₁ gets smaller when y < y_allowable as shown in Figure 7(b), and thus y tends to be larger than y_allowable. Note that y cannot get much smaller (or larger) than y_target because D₁ remains stable when y < y_target for minimization, and y > y_target for maximization.

Figure 7(c) shows the individual function D₂ for tuning at a certain level y_target. The function D₂ was expressed as:

The desirability functions are calculated for various sets of the factor levels x (e.g., x = [–0.5,+0.2, 0, …, 1]). The set x that maximizes the desirability is the optimal parameter set. For the optimization calculation, limited-memory quasi-Newton code for bound-constrained optimization (L-BFGS-B) is employed (the function “optim” in R language can do the optimization).

In the case of having multiple objective variables, multi-objective optimization is also possible. Instead of maximizing individual desirability, the total desirability

is maximized; where i is the iterator corresponding to n objective variables. If no parameter set can meet all the limitations, i.e., if some of the desirability functions are zero, then D_total is evaluated as 0. In such situation, broadening a bit the limitations (y_allowable) of some objective variables can be helpful to obtain an optimization result.

The optimization procedure is as follows.

1. Click on “Register model” as shown in Figure 5. The registered model shows up in the selector in Figure 8.
2. Click on “Set” button to define the purpose (minimize/maximize/target) and lower or raise the limits.
3. Click on “Maximize desirability” to optimize total desirability D_total. “Maximize desirability” needs to be clicked several times because optimization starts from different initial points and can lead to different optimal conditions.
4. For multiple output values, it is necessary to register all the models and set their purposes of optimization individually.

Quality control

DSDApp has been validated by using a six-factor DSD table and simulated observations. The DSD table includes columns A-D as real factors and E and F as fake factors as shown in Table 1 (same as the table in Figure 5). Then, we generated simulated data y using the predefined models (same as the equation in Figure 2).

The users can verify the app functionality by checking whether or not the model built based on observation y is similar to the predefined model (1).

Afterwards, the model below was built based on the generated data using DSDApp as shown in “Model Information” in Figure 5.

As shown in Figure 9, the coefficients of the built model (6) and the original model (1) are similar, but with slight differences. To confirm that these differences are negligible compared to the variation ɛ, another set of the input points was introduced and the prediction validity of the model was checked.

Table 2 shows the confirmation points at the edge of the experimental space, where the maximum prediction error is expected to be observed. Figure 10 shows the correlation between predicted values ŷ based on the model (6) (vertical axis) and the true values y generated by the original function (horizontal axis) for the same input parameters as in Table 2. The blue dots correspond to y and ŷ at the DSD points, and the red dots to the confirmation points. By using the values of y and ŷ, the residual sum of squares for the confirmation points is calculated as 0.156, which is smaller than the original standard error σ = 0.30. Thus, the constructed model (6) fits well with the true model (1). Nevertheless, one should be cautious when several quadratic effects and two-factor interactions are active because these effects in DSD sometimes have relatively strong correlations [1]. In such a case, several possible models should be evaluated in later experiments to see which model is the most useful and reliable.

Similar tests in this section can be done by users’ defined equations. For example, “test.R” generates CSV files (DSD8-with-Y1.csv, DSD8-with-Y2.csv) with a DSD table and sample data by a hand-made equation. Users can upload the generated CSV files in DSDApp, and then check if the built models are similar to user’s predefined models.

Operating system

Windows (tested on Windows 10)

Programming language

R 3.6.3 (R.4.0)

Additional system requirements

Users can access to DSDApp at https://my-first-dsd.shinyapps.io/DSDApp_ver2/ on a web browser app.

For local usage, Rstudio is needed. Open server.R or ui.R in Rstudio, then click on “Run App.”

Dependencies

R packages daewr, shiny, and shinythemes.

List of contributors

P.C. conceptualized the idea and, together with J.H., provided supervision. R.H. developed the app and carried out tests to check its correct operation. All authors were involved in the writing of the original and the revised manuscript.

Software location

Language

English