Random Trial Sequencing Can Lead to False Significant Factors? —— Five Steps for Randomization, Blocking, and Center Points in DOE

1. An Experiment That Reached the Opposite Conclusion

A die-casting company conducted a two-level, three-factor experiment to address the issue of high porosity rates: shot speed, mold temperature, and coating concentration. The experiment consisted of 8 trials plus 2 center points, totaling 10 trials. To "avoid disrupting production," the trials were scheduled according to the production sequence: the high shot speed combination was arranged for Monday morning, the high mold temperature combination for Wednesday, and the high coating concentration for Friday.

The results showed that shot speed had the greatest effect, with a P-value of 0.008. Based on this, the team increased the shot speed by 15%, but the porosity rate remained almost unchanged after production. It was only after a three-month review that they discovered the true differences came from factors that changed in sync with the trials: the die-casting machine had just been fitted with a new mold on Monday, the workshop air conditioning failed on Wednesday, and a new batch of coating was used on Friday. The trial sequence and batch differences were attributed to shot speed.

The calculations in DOE were correct, but the scheduling of the trials was flawed. This article discusses the three aspects of DOE that are most often overlooked but significantly impact the reliability of the conclusions: randomization, blocking, and center points.

2. Managing the Same Enemy: Noise

DOE has an implicit assumption: all conditions should be as consistent as possible except for the factors being intentionally changed, and any remaining differences are considered random errors. However, this is rarely the case in the field—raw materials come in different batches, there are multiple machines, different shifts, and environmental factors like temperature and humidity vary over time. These differences that cannot be included as experimental factors are known as noise factors. When they change in sync with the factors, they can become entangled with the factor effects, a phenomenon known statistically as confounding.

Each of the three aspects addresses a different part of the problem. Randomization prevents unknown, time-drifting noise from binding with the factors by shuffling the trial sequence, rather than arranging them by factor levels or following the design table order. Blocking isolates known, blockable noise from the error by separating it into distinct groups. For example, if a batch of material, a specific machine, or a particular shift cannot be eliminated within the experiment, they should be treated as blocks rather than ignored. Center points, where all factors are set at their mid-levels, serve two purposes: they test for curvature in the response and provide an estimate of pure error, which helps determine if the model fits well.

In summary: randomization disperses noise, blocking isolates noise, and center points quantify noise. Skipping any of these steps means the reliability of the conclusions will depend on luck.

3. Five-Step Method

Step 1: List Noise Sources Before Starting

Divide potential noise sources into two categories based on whether they can be blocked. Blockable sources include raw material batches, shifts, machines, operators, and sample racks, which have clear boundaries. Non-blockable sources include temperature drift, tool wear, and environmental humidity, which can only be managed by randomization.

Pay special attention to three common sources of noise: raw material batches (especially when mixing old and new materials), machines and cavities (especially multi-cavity molds), and time (morning vs. afternoon, beginning vs. end of the week). These are the most frequent in the field and are often conveniently scheduled within the same factor level.

Step 2: Determine Blocks First, Then Schedule Trials

The principle of blocking is to keep conditions as consistent as possible within each block while allowing differences between blocks. If a batch requires 8 trials and a shift can only complete 4, divide the trials into two blocks and ensure that each block contains a balanced combination of factor levels—do not let the first block consist entirely of low levels and the second block entirely of high levels.

The size of the blocks should align with natural production batches: one batch of raw materials, one furnace, one tray. Smaller blocks are better, but they should at least accommodate a complete set of factor level combinations.

Within each block, randomize the trial sequence. This can be done by drawing lots, using a random number table, or sorting with Excel's RAND(). The key is to document the random sequence, as it is part of the audit and traceability process.

Step 3: Distribute Center Points

Center points are typically conducted 3 to 5 times, and it is not recommended to do only one. More important than the number is the placement: center points should be distributed across different blocks and time periods to reflect both batch-to-batch differences and pure error. If all center points are completed on the first day, they will only capture the variation of that day.

Distinguish between true and false repetitions. True repetitions involve resetting, reprocessing, and remeasuring; false repetitions involve measuring the same sample multiple times. Only true repetitions can estimate pure error, while false repetitions only reflect the repeatability of the measuring instrument, which can inflate the significance of the factors. A related rule is that all sample measurements should be completed by a measurement system that has passed MSA, to avoid mixing measurement errors into the experimental conclusions.

Step 4: Include Blocks in the Analysis Model

After completing the trials, first examine the block effects. If the block effect is significant, it indicates that batch-to-batch differences do exist. In this case, the block must be retained in the model to account for this variation before assessing the significance of the factors. If the block term is removed, this difference will be entirely attributed to the error or directly to a specific factor.

Use center points to check for curvature and model fit. If the actual mean of the center points significantly deviates from the linear prediction at the same factor levels, it suggests that the response is curved. In this case, the two-level experiment should be stopped, and axial points should be added to create a response surface. If the center points align with the predictions, the two-level linear model is valid.

Also, examine the differences between repetitions, which represent pure error. Be cautious if the pure error is abnormally small, as this often indicates that the repetitions were not genuine and merely duplicated the same data.

Step 5: Clearly State the Applicable Conditions in the Conclusion

After analysis, the conclusion should not simply state "Factor A is significant." It should clearly specify three things: in which blocks the effect consistently appears, i.e., cross-block consistency; whether the optimal point falls within the experimental range or at the boundary; and the conditions under which the conclusion is valid, such as based on a specific grade of raw material or a particular machine model.

Archive the trial sequence table and block division. When conducting verification or scale-up experiments in the future, the same block structure can be reused to ensure comparability between the two sets of experiments.

4. Five Common Misconceptions

1. Treating Trial Sequence as Production Schedule

To avoid interrupting production, grouping trials with the same factor levels together is a common mistake. This self-created confounding is the most damaging to the conclusions.

2. Confusing "Random" with "Casual"

Randomization is the entry ticket for controlled experiments. Without randomization, the experiment is essentially just a set of observational data.

3. Partial Blocking

Running only one block through the complete design and filling the remaining combinations in other blocks can lead to contamination between block and factor effects.

4. One Center Point Is Enough

A single center point cannot reveal curvature or estimate error. Three is the minimum.

5. Using False Repetitions as Repetitions

Measuring the same sample multiple times will only treat measurement error as process error, making insignificant factors appear significant.

5. In Summary

The reliability of DOE conclusions depends not only on the correctness of the design table but also on the scheduling of the trials—noise must be dispersed, blocks isolated, and center points distributed. Only then do the data deserve to be analyzed with statistical models.

No matter how beautifully the factor effects are calculated, they cannot make up for the flaws in the trial scheduling.

Knowledge code: 6.4.1

Version: v20260920

Author: QTank QTank is dedicated to providing systematic professional knowledge, methodologies, and practical tools for quality management practitioners, helping companies continuously improve their quality capabilities.