A Visual Guide to Design of Experiments using Quantum XL

Developing Transfer
Functions Through
Design of Experiments
(DOE)
A Visual, Simplified Approach for
Performing DOE’s using Quantum XL
by Ray Balisnomo, Six Sigma Master Black Belt

Purpose of a DOE
To gain as much knowledge about a new or difficult design in the shortest
amount of time by:
 Quantifying which factors (X’s) are the most influential or important
 Deriving the mathematical relationship between the inputs and outputs
(Transfer Functions)
 Optimizing the design to best meet the customer’s requirements
90
93.75
97.5
101.25
105
108.75
112.5
116.25
120
100
120
140
160
180
200
220
240
260
Firing Angle (B)
DistanceY-Hat
Release Angle (A)
Distance Y-Hat
Surface Plot Release Angle (A) vs. Firing Angle (B)
240-260
220-240
200-220
180-200
160-180
140-160
120-140
100-120
200
212.5
225
237.5
250
262.5
275
287.5
300
80
100
120
140
160
180
200
220
240
260
90 93.75 97.5 101.25 105 108.75 112.5 116.25 120
Cup Elevation (C)
DistanceY-Hat
Firing Angle (B)
Distance Y-Hat
Surface Plot Firing Angle (B) vs. Cup Elevation (C)
240-260
220-240
200-220
180-200
160-180
140-160
120-140
100-120
80-100
7/26/2018 2

The Problem with DOE
Let K = number of factors in your experiment
 For linear models, you at least 2K observations.
 If you suspect non-linear relationships, you need 3K
Example: K = 6 factors
Number of Experiments = 64 x 3 replications = 192 observations for
linear; 2,187 for non-linear
7/26/2018 3
Problem: DOE’s can be very time a consuming and cost prohibitive
Question: What’s an efficient approach for conducting DOE’s?

Four Types of DOE Used in New Product Development*
1. Plackett-Burman (Main Effects Only)
2. Half-Fractional Designs (Only Main Effects & Two-way Interactions)
3. Full-Factorial Designs (All Possible Interactions)
4. Response Surface Design or RSD (Quadratic Equations)
*note: for Industrial Products
7/26/2018 4

Plackett-Burman Designs or Screening Designs
𝑌 = 𝑓 A, B, C = 𝛽0 + 𝛽1A + 𝛽2B + 𝛽3C + residuals
If K (the number of factors) = 6;
The number of required experiments, N, is at least 12*
You can only model Main
Effects (no interactions)
*note: assuming no replications
7/26/2018 5

Half Fractional Designs
7/26/2018 6
𝑌 = 𝛽0 + 𝛽1A + 𝛽2B + 𝛽3C +
𝛽4AB + 𝛽5BC + 𝛽6CA + residuals
• If K= 6; N ≥ 32*
• 167% more work than Plackett-Burman
You can model Two-way
Interactions and Main Effects

Full Factorial Designs
7/26/2018 7
𝑌 = 𝛽0 + 𝛽1A + 𝛽2B + 𝛽3C +
𝛽4AB + 𝛽5BC + 𝛽6CA + 𝛽7ABC + residuals
• If K= 6; N ≥ 64*
• 100% more work than Half Fractional
• 433% more work than Plackett-Burmann
You can model every possible
combination or interaction

Response Surface Design
7/26/2018 8
• If K= 6; N ≥ 90*
• 41% more work than Full Factorial
• 181% more work than Half Fractional
• 650% more work than Plackett-Burmann
You can model Quadratic
Relationships (i.e. X2)
𝑌 = 𝛽0 + 𝛽1A + 𝛽2B + 𝛽3C + 𝛽4AB + 𝛽5BC + 𝛽6CA
+ 𝛽7ABC + 𝛽8AA + 𝛽9BB + 𝛽10CC + residuals

Catapult: Teaching Aid for DOE
7/26/2018 9

Virtual Catapult:
7/26/2018 10
https://www.sigmazone.com/SL/Catapult/

7/26/2018 11
(Confidential – For Internal Use Only) Copyright © 2011 Rockwell Automation, Inc. All rights reserved. 11
The Text Book or White Box
Transfer Function for
Trajectory is in a form that
is not useful for predicting
the distance thrown by the
catapult. The input
variables or factors are not
in the same units as our
catapult’s controls.

Choose high & low
setting for K factors
(2 levels)
Use a full
factorial design
Use a half
fractional design
(resolution 5)
Use a Plackett-
Burman 12-run
design & filter-out
insignificant x’s
NO YES
6≤K≤11?
K>4?
YESNO
4
7
Is
the Model
linear?
Convert existing
DOE to Central
Composite Design
(CCD)
Reduce model and
change the
coefficients using Un-
coded Regression
Optimize Y using
Monte Carlo model
NO
Stop
YES
1 2
3
5
6
9
10
11
12
Start * Determine the
number of factors
(K) in the design
7
Add center
points to check
for curvature
Analyze DOE
and derive
linear Y=f(x)
8
Hypothesis Test (Accept Ho if P ≥ 0.10)
Ho: the value of the coefficient is zero.
Ha: the value of the coefficient is NOT zero.
*note: an adaption of the technique
taught by and credited to:
DOE Thought Process Map
7/26/2018 12

Parameter Diagram or P-Diagram
7/26/2018 13
Y = f(x)input output
a) Release
Angle
Noise variables:
Distance (cm)
Control variables:
 Measurement System Error
 Wind Resistance
 Set-up / Human Error
 Etcetera…
b
c
d
e
b) Firing Angle
c) Cup Elevation
d) Pin Elevation
e) Bungee Position
a
K = 5

Example of Measurement Error
7/26/2018 14
What should I
record as the
distance?
Is it 67, 67.5,
67.8, or 68?

Choose high & low
(2 levels)
Use a full
factorial design
Use a half
fractional design
(resolution 5)
Use a Plackett-
Burman 12-run
design & filter-out
insignificant x’s
NO YES
6≤K≤11?
K>4?
YESNO
4
7
Is
the Model
linear?
Convert existing
DOE to Central
Composite Design
(CCD)
Reduce model and
change the
coded Regression
Optimize Y using
Monte Carlo model
NO
Stop
YES
1 2
3
5
6
9
10
11
12
number of factors
(K) in the design
7
Add center
points to check
for curvature
Analyze DOE
and derive
linear Y=f(x)
8
7/26/2018 15
K = 5

Determine the Lowest and Highest Settings for Each Factor
7/26/2018 16
b
c
d
e
a
a
b
c
d
e
K = 5

Choose high & low
(2 levels)
Use a full
factorial design
Use a half
fractional design
(resolution 5)
Use a Plackett-
Burman 12-run
design & filter-out
insignificant x’s
NO YES
6≤K≤11?
K>4?
YESNO
4
7
Is
the Model
linear?
Convert existing
DOE to Central
Composite Design
(CCD)
Reduce model and
change the
coded Regression
Optimize Y using
Monte Carlo model
NO
Stop
YES
1 2
3
5
6
9
10
11
12
number of factors
(K) in the design
7
Add center
points to check
for curvature
Analyze DOE
and derive
linear Y=f(x)
8
DOE Though Process Map
7/26/2018 17
K = 5

QXL DOE Create Design Create 2-Level Factorial Design
7/26/2018 18

K = 5 Factors; Choose Half-Fractional Design (Resolution V)
7/26/2018 19

Enter Factor Names with Low and High Settings
7/26/2018 20
Note: Factors currently are in Coded Units (Low=-1, High=+1)

Enter Factor Names with Low and High Settings
7/26/2018 21
Note: Now Factors are in Un-Coded Units
Un-coded Unit = Physical Unit of Measure

Enter Output Name (e.g., Distance in Centimeters)
7/26/2018 22

Calculating: Number of Replicates*, c
7/26/2018 23




 1
32
N
c
Where,
c = Number of replications or repeats
N = Number of experimental runs
Round-up to the nearest integer *as taught by:

Calculating: Number of Replicates, c
7/26/2018 24




 1
32
N
c
Where,
c = Number of replications or repeats
N = Number of experimental runs
Round-up to the nearest integer
16
3 *as taught by:

Choose high & low
(2 levels)
Use a full
factorial design
Use a half
fractional design
(resolution 5)
Use a Plackett-
Burman 12-run
design & filter-out
insignificant x’s
NO YES
6≤K≤11?
K>4?
YESNO
4
7
Is
the Model
linear?
Convert existing
DOE to Central
Composite Design
(CCD)
Reduce model and
change the
coded Regression
Optimize Y using
Monte Carlo model
NO
Stop
YES
1 2
3
5
6
9
10
11
12
number of factors
(K) in the design
7
Add center
points to check
for curvature
Analyze DOE
and derive
linear Y=f(x)
8
7/26/2018 25

Definitions
• Blocking: a technique to quantify if a nuisance or noise variable is affecting
the output. For example: the operator or test technician
• Folding: a technique to reduce confounding. Confounding occurs when you
have a fractional factorial design and one or more effects cannot be
estimated separately.
7/26/2018 26

Add 3 Center Points per Block (Note: No Blocking, No Folding)
7/26/2018 27
Number of Replicates = Number of Center Points

Blank DOE Observation Sheet When Finished:
7/26/2018 28

Hide Tips to Improve Stability
7/26/2018 29
Duration: between 35 minutes and 45 minutes

Example* of DOE Observation Sheet with Response Variables
7/26/2018 30
*note: actual results may vary due to the impact of Noise Variables

Choose high & low
(2 levels)
Use a full
factorial design
Use a half
fractional design
(resolution 5)
Use a Plackett-
Burman 12-run
design & filter-out
insignificant x’s
NO YES
6≤K≤11?
K>4?
YESNO
4
7
Is
the Model
linear?
Convert existing
DOE to Central
Composite Design
(CCD)
Reduce model and
change the
coded Regression
Optimize Y using
Monte Carlo model
NO
Stop
YES
1 2
3
5
6
9
10
11
12
number of factors
(K) in the design
7
Add center
points to check
for curvature
Analyze DOE
and derive
linear Y=f(x)
8
7/26/2018 31

QXL DOE Analyze Design Run Regression
7/26/2018 32

Analyze Design: 3 Questions for Validating Transfer Function
1. Is the Model or Transfer Function Linear? If YES, we are done; if NO we
need to add more data points (e.g. convert the Fractional Design into a
Central Composite Design).
2. Are All the Factors Significant? The model could have terms that
artificially inflate the Correlation Coefficient or R-square. We need to
remove the in-significant terms or interactions before we can trust the final
R-square value.
3. How Trust-worthy is the Model or Transfer Function? We want the
highest R-square possible. The R-square values represents what
percentage of the changes to our response value can be explained by the
transfer function (e.g., if R-square = 85%, then the inputs or factors we
looked at can explain 85% of our output. There’s a 15% probability that our
output will not match what we actually observed).
7/26/2018 33

Choose high & low
(2 levels)
Use a full
factorial design
Use a half
fractional design
(resolution 5)
Use a Plackett-
Burman 12-run
design & filter-out
insignificant x’s
NO YES
6≤K≤11?
K>4?
YESNO
4
7
Is
the Model
linear?
Convert existing
DOE to Central
Composite Design
(CCD)
Reduce model and
change the
coded Regression
Optimize Y using
Monte Carlo model
NO
Stop
YES
1 2
3
5
6
9
10
11
12
number of factors
(K) in the design
7
Add center
points to check
for curvature
Analyze DOE
and derive
linear Y=f(x)
8
7/26/2018 34

1. Is The Model Linear?
7/26/2018 35
Two Conditions Which Must be TRUE for the Model to be Linear:
1) At the Center Point, the Predicted Value, Y-hat, is within the Lower and
Upper Predictive Interval (PI)
2) If you plot the Factors versus the Residuals, the value of the error should
be relatively stable at all levels (e.g., Low Setting, Center Point, High
Setting)
150
160
0
25
170
180
1
120
90
105
50
75
)
elgnAgniriF
elgnAesaeleR
100
90
100
150
200
105
1120
160
150
180
170
200
250
elgnAesaeleR
elgnAgniriF
on-linear Model (Central Composite DesignN o CCD) Linear Model (Half-Fractional Design)r
Central Composite Design
Half-fractional Design
Which One is the Actual Transfer Function?

1. Is the Model Linear? Condition 1
7/26/2018 36
PI = Predictive Interval
If PI Lower ≤ Y-Hat ≤ PI Upper ; then the Model is Linear
For example:
Y-Hat = 155.7
Since 152.4 ≤ Y-Hat ≤ 159.1; we can assume our equation is stable (there are no missing terms)

7/26/2018 37
172
182
192
202
212
222
232
242
95 97 99 101 103 105 107 109 111 113 115
Firing Angle versus Distance
Y-hat (predicted
value)
Upper PI
Actual Value
Firing Angle
Distance

7/26/2018 38
Model is NOT Linear. The transfer function has squared terms.
Model is linear because the average error is the same, but the variance is greater at the center than at the corners.

QXL DOE Charts  Residual Analysis
7/26/2018 39

2. Which Factors Are Significant?
• Null Hypothesis, Ho: The Coefficient is ZERO (it’s not significant)
• Alternative Hypothesis, Ha: The Coefficient is NOT ZERO (factor is important).
Therefore, we are hoping for low P-values.
• Accept Ho if P-value ≥ Acceptable Risk (Alpha-value)
• Typically, we set the Alpha value between 0.05 and 0.10 (5% - 10%)
• Uncheck the In-significant Terms (Do not include them in the Model)
7/26/2018 40
Uncheck the In-significant Terms
Uncheck the In-significant Terms

QXL DOE Charts  Pareto Regression Coefficients
7/26/2018 41

2. Which Factors are Significant?
7/26/2018 42
Ranked by Order of Importance

3. How Trustworthy is the Model?
Two Conditions Which Must be TRUE for the Model to be Trustworthy.
1. The R-squared value is high
a) The ideal R-squared value is greater than 95%, but an R-squared value as low as 85% is
considered actionable.
b) You must reduce the model to it’s most significant terms before you can evaluate the R-squared
value. A method for artificially inflating the R-squared value is to leave or add as many terms to the
model as possible.
2. There are no warnings in the model (e.g., the Residuals are normally
distributed)
7/26/2018 43

7/26/2018 44

7/26/2018 45

Choose high & low
(2 levels)
Use a full
factorial design
Use a half
fractional design
(resolution 5)
Use a Plackett-
Burman 12-run
design & filter-out
insignificant x’s
NO YES
6≤K≤11?
K>4?
YESNO
4
7
Is
the Model
linear?
Convert existing
DOE to Central
Composite Design
(CCD)
Reduce model and
change the
coded Regression
Optimize Y using
Monte Carlo model
NO
Stop
YES
1 2
3
5
6
9
10
11
12
number of factors
(K) in the design
7
Add center
points to check
for curvature
Analyze DOE
and derive
linear Y=f(x)
8
7/26/2018 46

QXL DOE Create Design Create CCD Design
7/26/2018 47

For 5 Factors, Choose Half Factorial Design (V)
7/26/2018 48

Substitute the Coded Units with Actual or Un-coded Units
7/26/2018 49

Give the Output a Name (Note: You can multiple outputs)
7/26/2018 50

Choose the Number of Replications
7/26/2018 51

Augmenting and Existing Factorial DOE to Accommodate Non-linear Terms
7/26/2018 52
Modified to
Full Factorial Design with
Center Points
Central Composite Design
(CCD) – Face Centered
1
1
11
1
1
1
0
0
1
1
11
1
1
1
0
0
-1
-1
-1
-1
-1
-1
1
1

Enter 1 Center Point and Choose: Face CCD Alpha=1
7/26/2018 53

An Infeasible DOE: A Sphere Outside the Cube
7/26/2018 54
Setting values are outside the High and Low limits.
(This experiment is impossible)

First 16 Runs is Half Fractional Design: CCD is Just 11 Additional Experiments
7/26/2018 55

Example* of Completed CCD Observation Sheet (27 x 3 replicates = 81 data pts.)
7/26/2018 56
*note: due to noise variable, the actuals results could be different

Target is 460cm ± 10 cm
• What should the settings be for Control Factors:
• Firing Angle?
• Cup Elevation?
• Pin Elevation?
• Bungee Position?
• How far back should we pull the catapult (Release Angle)?
• How trustworthy is the model or transfer function?
57
450cm – 470cm
7/26/2018

Enter the Upper Spec. Limit and Lower Spec. Limit in the Data Sheet
7/26/2018 58

Choose high & low
(2 levels)
Use a full
factorial design
Use a half
fractional design
(resolution 5)
Use a Plackett-
Burman 12-run
design & filter-out
insignificant x’s
NO YES
6≤K≤11?
K>4?
YESNO
4
7
Is
the Model
linear?
Convert existing
DOE to Central
Composite Design
(CCD)
Reduce model and
change the
coded Regression
Optimize Y using
Monte Carlo model
NO
Stop
YES
1 2
3
5
6
9
10
11
12
number of factors
(K) in the design
7
Add center
points to check
for curvature
Analyze DOE
and derive
linear Y=f(x)
8
7/26/2018 59

QXL DOE Analyze Design Run Regression
7/26/2018 60

Reduced Regression Model (Coded Units)
7/26/2018 61

Reduced Regression Model (Coded Units)
7/26/2018 62

What is the Variance Inflation Factor (VIF)?
• VIF is a measure of multicollinearity in a model. Multicollinearity occurs when inputs are correlated
with each other. Most DOEs will have VIFs=1 which indicate that the inputs are orthogonal
(independent). However, when working with regression models which originate from non-DOE
sources, the VIFs are likely to be larger than 1.
• As the VIFs increase from 1, the degree of multicollinearity increases. There is a great deal of
debate over how much multicollinearity is too much. Some experts suggest that values over 2 are
excessive while others draw the line at 10. Minitab© suggest the upper limit or cut-off point for
VIF is 5. According to Minitab© : “A VIF value greater than 5 suggests that the regression
coefficient is poorly estimated due to severe multicollinearity.”
7/26/2018 63
From Minitab18 Help Menu

Convert Regression Model from Coded Units to Un-coded Units
7/26/2018 64
Un-coded Units: The Factor’s actual or physical units of measure
Coded Units: Required in order to calculate P-values (Orthogonal Designs)

QXL DOE Analyze Design Uncoded Coefficients
7/26/2018 65

Un-coded Regression
7/26/2018 66
Note the change in the Coefficient Values

Choose high & low
(2 levels)
Use a full
factorial design
Use a half
fractional design
(resolution 5)
Use a Plackett-
Burman 12-run
design & filter-out
insignificant x’s
NO YES
6≤K≤11?
K>4?
YESNO
4
7
Is
the Model
linear?
Convert existing
DOE to Central
Composite Design
(CCD)
Reduce model and
change the
coded Regression
Optimize Y using
Monte Carlo model
NO
Stop
YES
1 2
3
5
6
9
10
11
12
number of factors
(K) in the design
7
Add center
points to check
for curvature
Analyze DOE
and derive
linear Y=f(x)
8
7/26/2018 67

Optimization Using Monte Carlo Simulation
1. Review  Unprotect Sheet: Prepare to modify Un-coded Regression
2. Create Textual Report of Residuals (Display the Actual Values)
3. Enter “0” in an empty cell and rename the cell “modeling_error”
4. Define an Input Variable and Fit a Distribution using Residual
Data
5. Define Model Settings as Input Variables with Constants
6. Add “Residuals” with Zero-value to Y-hat (Predicted Distance)
7. Define Y-hat as an Output Variable with USL and LSL
8. Optimize the Model with these settings:
a) Objective: Maximize the Cpk-value of Y-hat (use Output Name)
b) Constraint: Minimum of Y-hat (use Output Name) must be greater
than LSL
9. Verify Design with 10 samples (Capability Study)
7/26/2018 68

Data
than LSL
7/26/2018 69

Select Worksheet Un-coded Regression: Review  Unprotect Sheet
7/26/2018 70
Warning: Monte Carlo Simulation will fail if Un-coded Regression is Protected

Data
than LSL
7/26/2018 71

QXL DOE Charts  Residual Plots: Check the Box: Create Textual Report
7/26/2018 72
Check this Box

Textual Report of Residuals (see Column L)
7/26/2018 73

Data
than LSL
7/26/2018 74
Note:
It does not matter what you name the cell.
The purpose is to create an absolute cell
reference that can be referred to easily
inside a formula in another worksheet.

Enter the Number “0” in Cell L5 and Add a Label Next to It
7/26/2018 75
Enter the Value “0” in Cell L5
Y-hat = 𝛽0 + 𝛽1A + 𝛽2B + 𝛽3C + 𝛽4AB + 𝛽5BC + 𝛽6CA
+ 𝛽7ABC + 𝛽8AA + 𝛽9BB + 𝛽10CC + Residuals = 0
Cell L5 = 0
Cell K5 = “Residuals”

Right Mouse Click Cell L5 and Define Name
7/26/2018 76
*
* note:
You can give it any name. It doesn’t have to be “modeling_error”

Data
than LSL
7/26/2018 77

QXL Monte Carlo  Mark Input
7/26/2018 78
Note:
It’s important to keep your cursor on Cell L5

Click on the Button: Fit from data
7/26/2018 79

Select All the Residuals (Range L10:L90)
7/26/2018 80

Select the Distribution on Top (Best Fitting Shape)
7/26/2018 81
Note
1) The Goodness of Fit Test (P-value) sorts by Descending order
2) We are able to model Non-normal Distributions
Best Fit
Worst Fit

Model the Residuals based on Fitted or Actual Data
7/26/2018 82
Y-hat = 𝜷 𝟎 + 𝜷 𝟏A + 𝜷 𝟐B + 𝜷 𝟑C + 𝜷 𝟒AB + 𝜷 𝟓BC + 𝜷 𝟔CA
+ 𝜷 𝟕ABC + 𝜷 𝟖AA + 𝜷 𝟗BB + 𝜷 𝟏𝟎CC
+ Modeling_Error = 0
Logistic Distribution with
Location = 0
Scale = 1.88

Check the Box because
Residuals = White Noise

Data
than LSL
7/26/2018 83

Select Worksheet: Un-coded Regression and Enter Input Variables
7/26/2018 84
Select the Cell Range: F11:F15

Define Five Inputs as Constant Variables
7/26/2018
85
Scroll to the Bottom of the Drop Down Menu

Data
than LSL
7/26/2018 86

Add the Residuals to the Output (Y-hat) in Cell I10
7/26/2018 87
Enter Formula in Cell I10
Defined Name
from previous step
(Step #4)
Note: Worksheet Must be Unprotected
Formula Bar in Excel:
Worksheet: Uncoded Regression

Data
than LSL
7/26/2018 88

Define Monte Carlo Simulation Output
7/26/2018 8989
Warning:
Cell I10 Must be Selected
Before You Choose
“Mark Output”

Data
than LSL
7/26/2018 90

QXL Monte Carlo Optimize
7/26/2018 91

7/26/2018 92
Maximize the Capability (Cpk) of Predicted Distance
Optimizer: Define the Objective

7/26/2018 93
The Minimum of the Predicted
Distance must be Greater than
LSL = 450. This constraint
ensures that we will the target.
Optimizer: Define Constraint

7/26/2018 94
Note: This is different from setting up the high and
low settings for a DOE. This is establishing the
inference space for the solution.
Optimizer: Define the Ranges for the Input

Optimizer: Define the Defects Per Million (DPM)
7/26/2018 95

Optimized Set Points for Hitting the Target
7/26/2018 97

Data
than LSL
7/26/2018 98

QXL Stat Tools Analyze Tools Capability Analysis
7/26/2018 99

Run 10 Trials and Perform Capability Study
7/26/2018 100
Z-score = 3 x Cpk

Summary of Performing a DOE
1. Run a screening DOE if you have 6 or more factors.
2. If you have 4 or less factors run a full factorial design; otherwise, run a half-
fractional design
3. If curvature exists, augment your factorial design with axial points. Analyze
your DOE as a Central Composite Design Model.
4. Optimize your Design using Monte Carlo Simulation. This method is robust
to Model’s with non-normal residuals.
7/26/2018 101

The End
If you have questions, please e-mail: rbalisnomo@ra.rockwell.com
7/26/2018 102

A Visual Guide to Design of Experiments using Quantum XL

More Related Content

Similar to A Visual Guide to Design of Experiments using Quantum XL (20)

More from Ramon Balisnomo (6)

Recently uploaded (20)

A Visual Guide to Design of Experiments using Quantum XL