Statistical analysis and experimental designs

Experimental Design & Analysis
Lecture # 03

 To eliminate bias
 To ensure independence among observations
 Required for valid significance tests and interval estimates
Old New Old New Old New Old New
In each pair of plots, although replicated, the new variety is
consistently assigned to the plot with the higher fertility level.
Low High
Randomization

Replication
 The repetition of a treatment in an experiment
A A
A
B
B
B
C
C
C
D
D
D

Advantages of Replication
 Each treatment is applied independently to two or more
experimental units
 Variation among plots treated alike can be measured
 Increases precision - as n increases, error decreases
Sample variance
Number of replications
Standard error
of a mean
 Broadens the base for making inferences
 Smaller differences can be detected

Effect of number of replicates
Effect of replication on variance
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
4.5
5.0
5.5
6.0
6.5
7.0
7.5
8.0
0 5 10 15 20 25 30 35 40 45 50
number of replicates
Variance
of
the
mean

What determines the number of replications?
 Pattern and magnitude of variability in the soils
 Number of treatments
 Size of the difference to be detected
 Required significance level
 Amount of resources that can be devoted to the
experiment
 Limitations in cost, labor, time, and so on

Local control
Randomization and replication cannot control all the extraneous
sources of variation. Further steps are needed to improve the
experimental design. All such techniques which leads to the
improvement of the experimental design are called Local control
i.e. Blocking , block size and experimental unit size. The main
purpose of the local control is to increase the efficiency by
decreasing the experimental error.

Strategies to Control Experimental Error
 Select appropriate experimental units
 Increase the size of the experiment to gain more
degrees of freedom
– more replicates or more treatments
– caution – error variance will increase as more heterogeneous
material is used - may be self-defeating
 Select appropriate treatments
– factorial combinations result in hidden replications and therefore
will increase n
 Blocking
 Refine the experimental technique
 Measure a concomitant variable
– covariance analysis can sometimes reduce error variance

The Field Plot
 The experimental unit: the vehicle for evaluating
the response of the material to the treatment
 Shapes
– Rectangular is most common - run the long dimension parallel to
any gradient
– Fan-shaped may be useful when studying densities
– Shape may be determined by the machinery or irrigation

Plot Shape and Orientation
 Long narrow plots are preferred
– usually more economical for field operations
– all plots are exposed to the same conditions
 If there is a gradient - the longest plot dimension should
be in the direction of the greatest variability



Border Effects
 Plants along the edges of plots often perform differently
than those in the center of the plot
 Border rows on the edge of a field or end of a plot have an
advantage – less competition for resources
 Plants on the perimeter of the plot can be influenced by
plant height or competition from adjacent plots
 Machinery can drag the effects of one treatment into the
next plot
 Fertilizer or irrigation can move from one plot to the next
 Impact of border effect is greater with very small plots

Minimizing Border Effects
 Leave alleys between plots to minimize drag
 Remove plot edges and measure yield only on
center portion
 Plant border plots surrounding the experiment

Experimental Design
 An Experimental Design is a plan for the assignment of the treatments
to the plots in the experiment
 Designs differ primarily in the way the plots are grouped before the
treatments are applied
– How much restriction is imposed on the random assignment of treatments
to the plots
A B
C
D A
A
B
B
C
C
D
D
C
D
A B
A
A
B
B
C
C
D
D

Why do I need a design?
 To provide an estimate of experimental error
 To increase precision (blocking)
 To provide information needed to perform tests of significance and
construct interval estimates
 To facilitate the application of treatments - particularly cultural
operations

Factors to be Considered
 Physical and topographic features
 Soil variability
 Number and nature of treatments
 Experimental material (crop, animal, pathogen, etc.)
 Duration of the experiment
 Machinery to be used
 Size of the difference to be detected
 Significance level to be used
 Experimental resources
 Cost (money, time, personnel)

Cardinal Rule:
Choose the simplest
experimental design
that will give the
required precision
within the limits of the
available resources

Completely Randomized Design (CRD)
 Simplest and least restrictive
 Every plot is equally likely (has equal chance) to
be assigned to any treatment
A A
A
B
B
B
C
C
C
D
D
D

Advantages of a CRD
 Flexibility
– Any number of treatments and any number of replications
– Don’t have to have the same number of replications per treatment (but
more efficient if you do)
 Simple statistical analysis
– Even if you have unequal replication
 Missing plots do not complicate the analysis
 Maximum error degrees of freedom

Disadvantage of CRD
 Low precision if the plots are not uniform
A B
C
D A
A
B
B
C
C
D
D

Uses for the CRD
 If the experimental site is relatively uniform
(possess less variability)
 If a large fraction of the plots may not respond or
may be lost or damage during experimentation
e.g. Animals may die before the completion of the
experiment.
 If the number of plots is limited (the experimental
units are small) such as green house or laboratory
experiments.

Design Construction
 No restriction on the assignment of treatments to the
plots
 Each treatment is equally likely to be assigned to any
plot
 Should use some sort of mechanical procedure to
prevent personal bias (randomization)

Experimental Layout of CRD
The layout of the experiment is the actual placement of the
treatments on the experimental units. Suppose there are t
Treatments and the whole experimental material is divided into N
experimental units. Let each treatment is assigned to r
experimental units (replicates) such that rt= N, the total number of
experimental units. Usually each treatment is applied or
replicated equal number of times. experimental unit is

Randomization Procedure
Following are the steps used in randomization procedure,
1. Determine the total number of experimental units rt= N
2. Assign plot number to each experimental unit say 1 to N
3. Assign treatments to the experimental units randomly.
Assignment of random numbers may be by:
– lot (draw a number )
– computer assignment
– using a random number table

Random Assignment by Lot
 We have an experiment to test three varieties: the top line from
Oregon, Washington, and Idaho to find which grows best in our
area ----- t=3, r=4
1 2 3 4
5 6 7 8
9 10 11 12
A
A
A
A
12
1
5
6

Random Assignment by Computer (Excel)
 In Excel, type 1 in cell A1, 2 in
A2.
Block cells A1 and A2. Use the
‘fill handle’ to drag down through
A12 - or through the number of
total plots in your experiment.
 In cell B1, type = RAND(); copy
cell B1 and paste to cells B2
through B12 - or Bn.
 Block cells B1 - B12 or Bn, Copy;
From Edit menu choose Paste
special and select values
(otherwise the values of the
random numbers will continue to
change)

Random numbers in Excel (cont’d.)
 Sort columns A and B
(A1..B12) by column
B
 Assign the first
treatment to the first r
(4) cells in column C,
the second treatment
to the second r (4)
cells, etc.
 Re-sort columns A B
C by A if desired.
(A1..C12)

Rounding and Reporting Numbers
To reduce measurement error:
 Standardize the way that you collect data and try to be as
consistent as possible
 Actual measurements are better than subjective readings
 Minimize the necessity to recopy original data
 Avoid “rekeying” data for electronic data processing
– Most software has ways of “importing” data files so that you don’t
have to manually enter the data again
 When collecting data - examine out-of-line figures
immediately and recheck

An Example
 Consider an investigation into the formulation of a new
“synthetic” fiber that will be used to make ropes
 The response variable is tensile strength
 The experimenter wants to determine the “best” level of
cotton (in wt %) to combine with the synthetics
 Cotton content can vary between 10 – 40 wt %; some
non-linearity in the response is anticipated
 The experimenter chooses 5 levels of cotton “content”;
15, 20, 25, 30, and 35 wt %
 The experiment is replicated 5 times – runs made in
random order
L. M. Lye DOE Course 28

An Example
 Does changing the
cotton weight percent
change the mean
tensile strength?
 Is there an optimum
level for cotton
content?

The Analysis of Variance
 In general, there will be a levels of the factor, or a treatments,
and n replicates of the experiment, run in random order…a
completely randomized design (CRD)
 N = an total runs
 We consider the fixed effects case only
 Objective is to test hypotheses about the equality of the a
treatment means

 The name “analysis of variance” stems from a
partitioning of the total variability in the
response variable into components that are
consistent with a model for the experiment
 The basic single-factor ANOVA model is
2
1,2,...,
,
1,2,...,
an overall mean, treatment effect,
experimental error, (0, )
ij i ij
i
ij
i a
y
j n
ith
NID
  
 
 


   


 


Models for the Data
There are several ways to write a model for the data:
is called the effects model
Let , then
is called the means model
Regression models can also be employed
ij i ij
i i
ij i ij
y
y
  
  
 
  
 
 

 Total variability is measured by the total sum
of squares:
 The basic ANOVA partitioning is:
2
..
1 1
( )
a n
T ij
i j
SS y y
 
 

2 2
.. . .. .
1 1 1 1
2 2
. .. .
1 1 1
( ) [( ) ( )]
( ) ( )
a n a n
ij i ij i
i j i j
a a n
i ij i
i i j
T Treatments E
y y y y y y
n y y y y
SS SS SS
   
  
    
   
 
 
 

 A large value of SSTreatments reflects large differences in
treatment means
 A small value of SSTreatments likely indicates no
differences in treatment means
 Formal statistical hypotheses are:
T Treatments E
SS SS SS
 
0 1 2
1
:
: At least one mean is different
a
H
H
  
  


 While sums of squares cannot be directly compared to test the
hypothesis of equal means, mean squares can be compared.
 A mean square is a sum of squares divided by its degrees of
freedom:
 If the treatment means are equal, the treatment and error mean
squares will be (theoretically) equal.
 If treatment means differ, the treatment mean square will be
larger than the error mean square.
1 1 ( 1)
,
1 ( 1)
Total Treatments Error
Treatments E
Treatments E
df df df
an a a n
SS SS
MS MS
a a n
 
    
 
 

The Analysis of Variance is Summarized in a Table
 The reference distribution for F0 is the Fa-1, a(n-1) distribution
 Reject the null hypothesis (equal treatment means) if
0 , 1, ( 1)
a a n
F F  


ANOVA Computer Output (Design-Expert)
Response:Strength
ANOVA for Selected Factorial Model
Analysis of variance table [Partial sum of squares]
Sum of Mean F
Source Squares DF Square Value Prob > F
Model 475.76 4 118.94 14.76 < 0.0001
A 475.76 4 118.94 14.76 < 0.0001
Pure Error161.20 20 8.06
Cor Total636.96 24
Std. Dev. 2.84 R-Squared 0.7469
Mean 15.04 Adj R-Squared 0.6963
C.V. 18.88 Pred R-Squared 0.6046
PRESS 251.88 Adeq Precision 9.294

The Reference Distribution:

Graphical View of the Results
DESIGN-EXPERT Plot
Strength
X = A: Cotton Weight %
Design Points
A: Cotton Weight %
S
tr
e
n
g
th
One Factor Plot
15 20 25 30 35
7
11.5
16
20.5
25
2
2
2
2
2
2 2
2
2
2 2
2
2
2

Model Adequacy Checking in the ANOVA
 Checking assumptions is important
 Normality
 Constant variance
 Independence
 Have we fit the right model?
 Later we will talk about what to do if some of
these assumptions are violated

Model Adequacy Checking in the ANOVA
 Examination of
residuals
 Design-Expert generates
the residuals
 Residual plots are very
useful
 Normal probability plot
of residuals
.
ˆ
ij ij ij
ij i
e y y
y y
 
 
DESIGN-EXPERT Plot
Strength
Residual
N
o
rm
a
l
%
p
ro
b
a
b
ility
Normal plot of residuals
-3.8 -1.55 0.7 2.95 5.2
1
5
10
20
30
50
70
80
90
95
99

Other Important Residual Plots
DESIGN-EXPERT Plot
Strength
2
2
2
2
2
2
2
2
2
2
2
2
2
2
Predicted
Residuals
Residuals vs. Predicted
-3.8
-1.55
0.7
2.95
5.2
9.80 12.75 15.70 18.65 21.60
DESIGN-EXPERT Plot
Strength
Run Number
Residuals
Residuals vs. Run
-3.8
-1.55
0.7
2.95
5.2
1 4 7 10 13 16 19 22 25

Post-ANOVA Comparison of
Means
 The analysis of variance tests the hypothesis of equal
treatment means
 Assume that residual analysis is satisfactory
 If that hypothesis is rejected, we don’t know which
specific means are different
 Determining which specific means differ following an
ANOVA is called the multiple comparisons problem
 There are lots of ways to do this
 We will use pairwise t-tests on means…sometimes
called Fisher’s Least Significant Difference (or Fisher’s
LSD) Method

Design-Expert Output
Treatment Means (Adjusted, If Necessary)
Estimated Standard
Mean Error
1-15 9.80 1.27
2-20 15.40 1.27
3-25 17.60 1.27
4-30 21.60 1.27
5-35 10.80 1.27
Mean Standard t for H0
Treatment Difference DF Error Coeff=0 Prob > |t|
1 vs 2 -5.60 1 1.80 -3.12 0.0054
1 vs 3 -7.80 1 1.80 -4.34 0.0003
1 vs 4 -11.80 1 1.80 -6.57 < 0.0001
1 vs 5 -1.00 1 1.80 -0.56 0.5838
2 vs 3 -2.20 1 1.80 -1.23 0.2347
2 vs 4 -6.20 1 1.80 -3.45 0.0025
2 vs 5 4.60 1 1.80 2.56 0.0186
3 vs 4 -4.00 1 1.80 -2.23 0.0375
3 vs 5 6.80 1 1.80 3.79 0.0012
4 vs 5 10.80 1 1.80 6.01 < 0.0001

For the Case of Quantitative Factors, a
Regression Model is often Useful
DOE Course 45
Response:Strength
ANOVA for Response Surface Cubic Model
Analysis of variance table [Partial sum of squares]
Sum of Mean F
Source Squares DF Square Value Prob > F
Model 441.81 3 147.27 15.85 < 0.0001
A 90.84 1 90.84 9.78 0.0051
A2
343.21 1 343.21 36.93 < 0.0001
A3
64.98 1 64.98 6.99 0.0152
Residual 195.15 21 9.29
Lack of Fit 33.95 1 33.95 4.21 0.0535
Pure Error 161.20 20 8.06
Cor Total 636.96 24
Coefficient Standard 95% CI 95% CI
Factor Estimate DF Error Low High VIF
Intercept 19.47 1 0.95 17.49 21.44
A-Cotton % 8.10 1 2.59 2.71 13.49 9.03
A2
-8.86 1 1.46 -11.89 -5.83 1.00
A3
-7.60 1 2.87 -13.58 -1.62 9.03

The Regression Model
Final Equation in Terms
of Actual Factors:
Strength = 62.611 -
9.011* Wt % +
0.481* Wt %^2 -
7.600E-003 * Wt %^3
This is an empirical model
of the experimental
results
DESIGN-EXPERT Plot
Strength
X = A: Cotton Weight %
Design Points
15.00 20.00 25.00 30.00 35.00
7
11.5
16
20.5
25
A: Cotton Weight %
Strength
One Factor Plot
2
2
2
2
2
2 2
2
2
2 2
2
2
2

DESIGN-EXPERT Plot
Desirability
X = A: A
Design Points
15.00 20.00 25.00 30.00 35.00
0.0000
0.2500
0.5000
0.7500
1.000
D
e
s
ir
a
b
ility
One Factor Plot
6
6
6
6
6
6
6
5
5
5
5
5
5
5
5
5
5
5
5
5
5
5
6
6
6
6
6
Predict 0.7725
X 28.23

Statistical analysis and experimental designs

More Related Content

Similar to Statistical analysis and experimental designs (20)

More from FarhanAhmad230 (17)

Recently uploaded (20)

Statistical analysis and experimental designs

Editor's Notes