Advancing QE Skills (21) | Transitioning Test Results to Mass Production: Verification, Validation, and Standardization

1. Parameters Copied Word for Word, but the Third Machine Produces a Batch of Cold Welds

A manufacturing company's welding process has long been plagued by cold welds. The QE team conducted a full factorial experiment, identified three significant factors, and found an optimal parameter set: welding current 4.2 kA, welding time 180 ms, and electrode pressure 2.6 kN. A trial production of 200 pieces passed the full inspection, and the QE team recorded these three values in the work instruction, signed it, and archived it, declaring the experiment complete. One month later, a new machine of the same model arrived in the workshop. The operator set the parameters according to the same work instruction, but the first piece had a cold weld. Soon after, batch variations began to appear on the old machine, and the operator adjusted the pressure to 3.0 kN, citing "it feels better this way" as the reason. The work instruction became a mere formality.

A post-mortem analysis revealed two major flaws: first, the "reproducibility verification" only involved 200 pieces, one machine, and one shift; second, the document only recorded a single point value without an allowable range, leaving operators to adjust parameters based on experience when there were slight deviations. The experiment itself was not flawed, but the mistake was treating the "conclusion in the report" as "already implemented capability."

2. Key Principles: Implementable Conclusions Are "Point + Range + Conditions"

The output of DOE is often written as an optimal combination, but a transferable conclusion must consist of three parts: central conditions (optimal settings for each factor), effective range (the range of factor values within which the response still meets specifications, derived from the response model at the specification boundaries), and applicable conditions (equipment, batch, environment, and operation methods not covered in the experiment but essential for the conclusion to hold).

Statistically, the results obtained during the trial phase are estimates of "repeated measurements under experimental conditions," with their variation described by the within-group standard deviation σ_within. The mass production phase introduces additional variation σ_between due to changes in equipment, batch materials, and shifts. The essence of whether the conclusion can be transferred is to determine if the σ_between in mass production is significantly greater than the experimental assumption—once it increases and the working point is close to the specification boundary, the "optimal" in the experiment will evaporate in mass production.

Therefore, the three steps of the transition each have their roles: verification answers "can the conclusion be reproduced," validation answers "can it meet standards under real mass production conditions," and standardization answers "can it be maintained when personnel and shifts change tomorrow." If any of these steps are missing, the conclusion remains in the report.

3. Five Practical Steps and Quantitative Criteria

Step 1: Classify factors to decide which to lock and which to release.

• Sensitive factors: If a change of 1 minimum adjustment unit within the experimental range results in a response change ≥ 10% of the specification width, it must be locked and poka-yoke implemented.
• Tolerant factors: If a change across the entire experimental range results in a response change ≤ 20% of the specification width, it can be relaxed to manual adjustment.
• Noise factors: Uncontrollable, and can only be managed by tightening central conditions or robust design to reduce sensitivity.

Criterion: After classification, a "factor → control item" table is formed. Each significant factor must appear in the table and fall under one of the three actions: lock, relax, or monitor. Any factor without an action is a conversion loophole.

Step 2: Design verification trials, with sample sizes stratified by characteristic importance.

• General characteristics: ≥ 3 batches, from different production days.
• Key characteristics: ≥ 5 batches, covering at least 2 machines and 2 shifts.
• Safety/regulatory characteristics: ≥ 5 batches, each batch ≥ 30 pieces, and covering 2 shifts.

Verification batches must be produced consecutively without interruption or special maintenance or preheating, and personnel should be assigned according to normal work schedules, not dedicated monitoring. Criterion: The time distribution of verification batches must not be concentrated in the same shift but should cover the actual mass production time distribution.

Step 3: Use four quantitative criteria for acceptance, focusing on distribution rather than pass rate.

• Central consistency: The difference between the mean of the verification batches and the optimal value from the experiment ≤ 0.5 × the within-group standard deviation of the experiment; exceeding this indicates uncontrolled condition differences.
• Variability consistency: The between-batch standard deviation of the verification batches ≤ 1.5 × the within-group standard deviation of the experimental phase.
• Model fit: All verification points fall within the prediction interval of the response model; if 1 out of 5 points falls outside, conduct 3 additional points and re-evaluate; if any points still fall outside, the transition is deemed a failure.
• Capability threshold: Cpk ≥ 1.33 for general characteristics, Cpk ≥ 1.67 for key/safety characteristics; simultaneously calculate the 95% confidence lower limit, which should not be less than 1.00 (sample size ≥ 30 or subgroup number ≥ 25).

All four criteria must be met for the conclusion to be considered reproducible. If any one is not met, return to the experimental phase to conduct more detailed trials, rather than proceeding to mass production with uncertain conclusions.

Step 4: Standardize documents: change point values to ranges and write ranges into the control plan.

• Work instructions should always be written as "central value ± allowable deviation," with upper and lower limits and locking methods (parameter permissions, tooling limits, poka-yoke devices) specified.
• Allowable deviation is derived from capability: find the factor value boundary that reduces Cpk to 1.33, and take 1/3 to 1/2 of the distance from this boundary to the central value as the allowable deviation for work instructions; set a lower limit—the range width must not be less than 3σ of the equipment repeatability, otherwise, there is no range.
• Synchronize updates to the control plan (control items, frequency, recording methods, reaction plans), PFMEA (occurrence and detection scores), first article inspection forms, and operator training lists.
• Post-training assessment: operators and team leaders must be able to independently state 2-3 locked items and their consequences of deviation at their workstations, with a 100% pass rate; those who fail cannot be scheduled.

Criterion (self-check for document consistency): randomly select 1 mass production batch and verify on-site that the actual parameters, control plan, and work instructions are completely consistent; any inconsistency is considered a failure of standardization.

Step 5: Set up mass production monitoring points and pre-agree on exit and rollback conditions.

Monitoring points are divided into two layers: process parameters (leading indicators, such as temperature, pressure, and torque, for early warning) and product characteristics (resulting indicators, such as dimensions and strength).

• Frequency: first article plus sampling inspection of 5 pieces every 2 hours (or per shift); key characteristics are sampled per batch.
• Intensification period: for the first 30 days after the transition, the sampling frequency is increased to three times the normal rate, with parameter verification and equipment inspection conducted every 5 days.
• SPC start rule: key characteristics are plotted on a control chart; after 25 consecutive groups without out-of-bounds or trend patterns, the frequency can be reduced to normal; if 1 point is out-of-bounds, 8 points are on the same side, or 6 points are monotonic, the reaction plan is initiated.
• Exit and rollback: if 3 consecutive batches show the same directional deviation, or the Cpk falls below 1.33 (key characteristics below 1.67), the process should be restored to the verification state within 24 hours and re-evaluated. The rollback value for parameters must be specified in the standardization phase and cannot be decided on the spot.

4. Five Common Pitfalls

Pitfall 1: Treating a single successful trial production as "verification complete." A trial production of 200 pieces passing full inspection only proves that "this batch can be produced," but it does not cover the variations introduced by different equipment, batches, and shifts. Typical issues include the absence of batch numbers, cross-equipment data, and shift coverage in the verification records; verification should be redone according to the batch composition in Step 2.

Pitfall 2: Writing optimal parameters as single point values. Single point values imply the assumption that they must be precise to the decimal place, while the equipment repeatability itself may cover the entire range. Once deviations occur due to environmental temperature differences or equipment wear, operators have no legitimate adjustment space and can only modify parameters based on feel—this is when the document becomes ineffective.

Pitfall 3: Only changing the work instruction, not the control plan or PFMEA. Parameters change, but the monitoring items and frequency in the control plan remain the same, and the occurrence and detection scores in the PFMEA are not updated. During audits, the three documents contradict each other. Criterion: before the change takes effect, the consistency of the three documents must be signed off by the QE for each item.

Pitfall 4: Setting up all monitoring points. Measuring dozens of parameters every half hour results in a pile of data that no one analyzes, and signals that arise go unresponded. Criterion: each monitoring item in the control plan must have a corresponding reaction plan and responsible person; monitoring items without a reaction plan should be deleted.

Pitfall 5: Treating the transition to mass production as the endpoint, without rollback and re-evaluation triggers. Experimental conclusions have applicable boundaries, and changes in material sources, major equipment repairs, mold replacements, and seasonal changes can all invalidate these boundaries. Four scenarios must trigger re-evaluation: changes in equipment or tooling for significant factors, changes in raw material suppliers or specifications, changes in operation methods, and 3 consecutive batches showing the same directional deviation or capability indices falling below the threshold. Often, the original conclusion is only recognized as inapplicable after problems arise, by which time the cost of correction has multiplied several times.

5. Self-Check List

• All significant factors are included in the "factor → control item" table with clear actions (lock, relax, monitor), no omissions.
• Verification batches meet stratified sample size requirements (general ≥ 3 batches, key ≥ 5 batches including 2 machines and 2 shifts, safety ≥ 5 batches × 30 pieces), produced consecutively without interruption, and covering different shifts.
• All four criteria are met: central difference ≤ 0.5σ, between-batch standard deviation ≤ 1.5σ, verification points within the prediction interval, and Cpk meeting standards (general ≥ 1.33, key ≥ 1.67, confidence lower limit ≥ 1.00).
• Work instructions are expressed as "central value ± allowable deviation," with allowable deviation derived from capability and not less than 3σ of equipment repeatability; control plan, PFMEA, first article inspection forms, and training records are updated and consistent with on-site checks.
• Monitoring points are divided into leading and resulting indicators with specified frequencies and reaction plans; the first 30 days involve intensified monitoring, and rollback values and four re-evaluation triggers are documented.

The report gives the conclusion, but mass production is the real test.

Knowledge code: 6.4.3

Version: v20261001

Author: QTank QTank is dedicated to providing systematic professional knowledge, methodologies, and practical tools for quality management practitioners, helping enterprises continuously improve their quality capabilities.