Bad Actor Analysis

Operational Excellence
Bad Actor Analysis
Introduction
3/11/2017 Ronald Morgan Shewchuk 1
• Sometimes our process goes awry and the root cause is blatantly obvious.
• Since the root cause is established, countermeasures can be implemented to
prevent recurrence of the out of control process condition.
• Other times the root cause is not apparent and investigations must be conducted
to drill down to the root cause(s).
• One-way ANOVA (Analysis of Variance) is a useful tool for identifying the Bad Actor
in your process.
• The ANOVA technique was invented by Sir Ronald Aylmer Fisher, a British
mathematician, in the 1920s.
• ANOVA conducts tests of hypotheses to determine if the means of three or more
populations are different.
• If we were only comparing two populations of data we would choose a 2-Sample
t -Test to analyze the data.
• Let us visit Niels Kohr in Case Study XXI to see a practical application of one-way
ANOVA for Bad Actor Analysis.

Bad Actor Analysis
Case Study XXI: One-Way ANOVA Analysis of Aluminum Extrusion Coating
Background
Niels Kohr, process engineer at a local aluminum extrusion company, was putting the finishing touches on his mental agenda for the
day during his morning commute to work. As he opened the front door to his manufacturing building, the carefully planned agenda
vaporized. Todd Larssen, the General Manager of the company, was waiting for him.
“Niels, we have a problem in the coating area. All the coating lines are down due to peel strength failure! Can you help us?”
Neils dropped off his lunch box, grabbed his hard hat, safety glasses and coveralls and headed out to the coating area. Indeed, all ten
coating lines were down. He interviewed the coating area Supervisor and Operators to get their assessment of the chain of events
that had led to the shutdown. Neils inspected the equipment on each coating line for any abnormality which would cause peel
strength failure. He scrolled back through the Human Machine Interfaces (HMI) for each controller to search for process upset
conditions. After several hours, he was unable to find an assignable cause for the low peel strengths.
Niels returned to his office and downloaded the process equipment operating data from the Data Historian on the server and the
associated QC test data from the Laboratory Information Management System (LIMS) for the last four days. His analysis steps and
rationalizations are captured in the following slides.

Bad Actor Analysis
Download process equipment operating data from Data Historian and the associated QC test data from the Laboratory Information
Management System (LIMS) for the last four days to Excel.

Bad Actor Analysis
Copy and paste the data from Excel to a new worksheet in Minitab.

Bad Actor Analysis
Step 1: Look at the Big Picture. Generate Individuals SPC Chart for Peel Strength for all Coating Lines. Click on Stat  Control Charts 
Variables Charts for Individuals  Individuals on the top menu.

Bad Actor Analysis
Select C14 Peel Strength (gm force) as Variables in the dialogue box. Click Scale.

Bad Actor Analysis
Click the toggle button for Stamp. Double Click C2 Time. Then Double Click C1 Date. Click Gridlines tab.

Bad Actor Analysis
Check the box for Y major ticks. Click OK. Then click OK one more time.

Bad Actor Analysis
The Individuals SPC Chart is generated for Peel Strength. Outliers are highlighted with red squares.

Bad Actor Analysis
Click the brush icon on the top menu. Left click and drag the box to highlight the low outliers. Let’s see what is driving these outliers.

Bad Actor Analysis
Return to the active worksheet. Scroll down until the brushed columns appear. These will have black dots next to the row numbers. There
is nothing here that immediately catches Niels’ attention. He elects to perform One-way ANOVA analysis on the data by Coating Line.

Bad Actor Analysis
Click Assistant  Hypothesis Tests on the top menu.

Bad Actor Analysis
Since we have ten coating lines. Click Help Me Choose under Compare more than two samples.

Bad Actor Analysis
The Peel Strength data is continuous and Niels wants to compare the means of the coating lines. Click One-Way ANOVA.

Bad Actor Analysis
Select Y data are in one column, X values in another column for data arrangement in the worksheet. Select C14 Peel Strength (gm force) for
Y data column and C3 Line No. for X values column. Leave the significance level, Alpha at 0.05. Click OK.

Bad Actor Analysis
Four reports are generated by the One-Way ANOVA analysis. This is the Summary Report which indicates that there are no differences
between the Peel Strength means by coating line.

Bad Actor Analysis
This is the Diagnostic Report which indicates that the majority of coating lines have peel strength outliers.

Bad Actor Analysis
This is the Power Report which indicates individual statistics by coating line and the ability to detect mean differences.

Bad Actor Analysis
This is the Report Card which indicates warnings and information about unusual data points, sample size, normality and individual
population variances.
i
!
i
!Data
Unusual
data entry or measurement errors. Consider removing data that are associated with special causes and repeating the analysis.
Diagnostic Report. You can hover over a point or use Minitab’s brushing feature to identify the worksheet row. Correct any
influence on the results, you should try to identify the cause of their unusual nature. These points are marked in red on the
Some of the data points are unusual compared to the others in the same sample. Because unusual data can have a strong
Size
Sample
enter a value for the difference.
To determine how large your samples need to be to detect a difference that has practical implications, repeat the analysis and
detection is sufficient. If this is your case, you can conclude that it is unlikely that there are any differences of 112.9 or larger.
least a 90% chance of detecting a difference of 112.9 between any two means. Some practitioners feel that an 80% chance of
having sample sizes that are too small. The Power Report shows that, based on your sample sizes and α, you would have at
Your data does not provide sufficient evidence to conclude that there are differences among the means. This may result from
Normality
normality cannot be reliably checked with small samples, you should use caution when interpreting the test results.
may be inaccurate with small samples. In addition, unusual data can have a strong influence on the test results. Because
Because some sample sizes are less than 20, normality can be an issue. If the data are not normally distributed, the p-value
Variance
Equal
shows that the test performs well with unequal variances, even when the sample sizes are not equal.
Minitab’s Assistant uses Welch’s method, which does not assume or require that the samples have equal variances. Research
Check Status Description
One-Way ANOVA for Peel Strengt by Line No.
Report Card

Bad Actor Analysis
Niels has not found the smoking gun. He returns to the active worksheet. Niels rationalizes that since coating lines are not assigned to any
specific tensile tester, there should be no peel strength differences between peel strength test instruments. He proceeds to check.

Bad Actor Analysis
Click Assistant  Hypothesis Tests on the top menu.

Bad Actor Analysis
Click One-Way ANOVA in the dialogue box.

Bad Actor Analysis
Replace Line No. with Tensile Tester No. in the X values column. Click OK.

Bad Actor Analysis
Tensile Tester No. 292 is indicated as a Bad Actor in the Summary Report. It has significantly lower mean peel strength vs the other Tensile
Test Instruments.

Bad Actor Analysis
Tensile Tester No. 292 has had a mean shift down in measured peel strength.

Bad Actor Analysis
Niels brushes the point at the beginning of the mean shift and finds the corresponding worksheet row to be 122.

Bad Actor Analysis
The black dot at row number 122 indicates that the problem with Tensile Tester No. 292 started on Feb 17 at 01:33. This allows Niels to
define the scope of the nonconformity.

Bad Actor Analysis
The Power Report indicates not only a low mean but also a high standard deviation due to the mean shift in Peel Strength for Tensile Tester
No. 292.

Bad Actor Analysis
The Report Card indicates no warnings. Niels can be confident in his conclusions about Tensile Tester No. 292.
i
Data
Unusual There are no unusual data points. Unusual data can have a strong influence on the results.
Size
Sample The sample is sufficient to detect differences among the means.
Normality
sample sizes are large enough.
Because all your sample sizes are at least 15, normality is not an issue. The test is accurate with nonnormal data when the
Variance
Equal
shows that the test performs well with unequal variances, even when the sample sizes are not equal.
Minitab’s Assistant uses Welch’s method, which does not assume or require that the samples have equal variances. Research
Check Status Description
One-Way ANOVA for Peel Strengt by Tensile Test
Report Card

Bad Actor Analysis
Click on the Show Graphs Icon on the top menu. Highlight all graphs. Right click over the highlighted graphs and select Send to Microsoft
PowerPoint. This produces a slide show presentation which will allow Niels to share with the production team.

Bad Actor Analysis
Summary
Niels presented his findings to the production team including the QC Manager, Elsa Anderssen. Elsa offered to begin daily SPC
monitoring of the tensile testing instruments using calibrated weights on the load cells. She also agreed to modify the Response Flow
Checklist for Non-conforming Material to validate peel strength results with another Tensile Tester before declaring the production
sample as discrepant.
There were high-fives all around the table and the production group was delighted to get back into business. As the group filed out of
the conference room Todd Larssen slapped Neils on the back and said,
“That was some fine detective work Neils! How about we go out to celebrate with a beer?”
It had been a long day. Neils paused for a moment and then responded,
“I think I would rather go out for ice cream…”

Bad Actor Analysis

More Related Content

What's hot (20)

Viewers also liked (11)

Similar to Bad Actor Analysis (20)

Recently uploaded (20)

Bad Actor Analysis