Michael Schuldenfrei, CTO
Leveraging Test Data
for Quality
GSA Quality Team Meeting
December, 2014
© Optimal+ 2014 2
The Need
Shifting from “Defects per Million” to “Defects per Billion”
© Optimal+ 2014 3
The Problem
No Problem Found
32%
Fab Process
28%
Test Program
10%
Test Operation
4%
Test Equipment
26%
RMA Source
No Problem Found Fab Process Test Program Test Operation Test Equipment
© Optimal+ 2014 4
The Challenge
BIGDATAEXPERTISE
COSTTIME
© Optimal+ 2014 5
Big Data – Device DNA
ECID
ECID
ECID
ECID
ECID
WAT
WS1
WS2
WAT
WS1
WS2
WAT
WS1
WS2
WAT
WS1
WS2
WS3
WAT
WS1
WS2
WS3
FT1
Burn in
FT2
Example: One package contains:
5 dice
x ~2 WS operations per die
x ~1.2 iterations per operation
x 3000 parametric measurements
+ 1000 per-site WAT measurements
+ 3000 FT measurements
A DNA consisting ~35K measurements!
An SLT lot with 5000 parts could have 150M historical
measurements from hundreds of wafers & FT lots
© Optimal+ 2014 6
Back to Basics
THE QUALITY QUESTION;
IS “GOOD” REALLY GOOD?
Outlier Detection
Geographic
Parametric
Escape Prevention
Test program issues
ATE issues
Data Feed Forward
(More intelligent decision making)
Drift
Smart Pairing
© Optimal+ 2014 7
Quality Solutions
Outlier Detection
© Optimal+ 2014 – All rights reserved 8
Optimal+ 2014 Company Confidential 9
Outlier Detection – Algorithms
D-PAT: Dynamic Part Average Testing
NNR: Nearest Neighbor Residual
Z-PAT: Z-Axis Part Average Testing
GDBN: Good Die in Bad Neighborhood
Zonal: Low yield zone-based detection
Final Test
Post Final-Test operation and Based on Die-ID (ECID etc.)
In real-time at Final-Test operation without Die-ID
Optimal+ 2014 Company Confidential 10
Cross-Operation Outlier Detection
Cross-operational quality based on Die ID
Contributing operations
ETEST/PCM/WAT
Wafer Sort
Final-Test
Burn-In
System Level Test
Example: E-Test based bin-switching performed post-Wafer Sort
The ability to identify potential bad devices based on E-test data
geographical analysis
Bin switching occurs post-wafer sort
Requires data-feed-forward within the supply chain
Escape Prevention
© Optimal+ 2014 – All rights reserved 11
© Optimal+ 2014 12
Escape Prevention – ATE Freeze
A freeze occurs when a tester instrument becomes “stuck”
and repeatedly returns the same or similar result for a
sequence of parts
© Optimal+ 2014 13
Escape Prevention – ATE / TP
13
The STDF “PRR.NUM_TESTS” field tells us the number of tests executed on
the part. It should be relatively stable throughout the lot
© Optimal+ 2014 14
Escape Prevention – Test Ops
Excessive probing – when operation ignores probe mark spec for a device
and keeps on probing to get the yield
© Optimal+ 2014 15
Escape Prevention – Test Program
Human error is one of the main contributors for test escapes
and RMA. Here the PE commented a few blocks in the TP for
debug and forgot to uncomment before production release:
Traditional SBL is design to detect yield issues in which a specific bin count
spikes. However human error can result in a drop to 0 which is missed.
SBL
SBL drop of soft bin 11
from ~3% to 0 following
new TP revision
© Optimal+ 2014 16
Escape Prevention – Test Program
Extremely loose test limits may mask real test performance problems
~95 Sigmas
~95 Sigmas
Advanced Quality
Solutions
© Optimal+ 2014 – All rights reserved 17
Implementations:
Within the same test area (e.g. WS, FT, etc.)
Between test areas (e.g. from WAT to WS to FT)
Within a single subcon
Between multiple subcons (hub and spoke)
Real-time (test program integration)
Offline bin-switching
Example scenarios:
Outlier Detection – drift analysis
Pairing – cherry-picking for power & speed combinations
Test program tuning
SLT / Burn-in reduction
© Optimal+ 2014 18
Data Feed Forward
© Optimal+ 2014 19
Data Feed Forward – Drift
Database at subconTester
1. ECID Data
2. FT1 Measurements
Test Program running
FT2 operation
Real-time data!
No test time impact!
One or more numeric values
representing the perceived
quality of a part based on:
Wafer geography
(e.g. edge vs. center)
Outlier detection rule inputs
(e.g. GDBN, Z-PAT, D-PAT, etc.)
Number of iterations to PASS
Overall lot/wafer yield
Equipment health during test
Parametric test results from
multiple operations
Etc…
© Optimal+ 2014 20
Quality Index
Quality Index
Lot/Wafer
Yield etc.
Quality
Rule
Inputs
Wafer
Geography
© Optimal+ 2014 21
“No Problem Found”
Combinations of chips causing issues:
IC3
IC2
PCB
IC1
© Optimal+ 2014 22
Smart Pairing
• New methodology to pair IC’s for
optimal compatibility
• Customer and suppliers agree on
recipe for “Best Match” between
IC’s (e.g. based on power
consumption and speed)
• “Quality Index” created based on
manufacturing and test data to
categorize chips
• Data fed-forward to assembly to
ensure IC’s pre-sorted into
“buckets” based on Quality Index
• MCPs and boards are assembled
with well-matched components
Grade A
Grade B
Grade C
Grade A
Grade B
Grade C
Supreme Quality requires a comprehensive
end-to-end approach which takes into
account problems arising from:
• Equipment
• Test Process
• Human Error
• Material
…and much more
© Optimal+ 2014 23
Conclusions
Q&A
Optimal+ 2014 Company Confidential 24

More Related Content

PDF
Breakthrough in Quality Management
PDF
ITC 2015 - Marvell Present : "Improving Quality and Yield Through Optimal+ Bi...
PPTX
Securing your supply chain from counterfeit parts through real time electroni...
PDF
AMD at ITC 2014
PDF
Leveraging Cross-Operational Test Data for Manufacturing Yield and DPPM/RMA I...
PDF
Bosch ConnectedWorld 2017: Striving for Zero DPPM
PPT
Use Models for Extending IEEE 1687 to Analog Test
PDF
Respond flow chart (rfc)
Breakthrough in Quality Management
ITC 2015 - Marvell Present : "Improving Quality and Yield Through Optimal+ Bi...
Securing your supply chain from counterfeit parts through real time electroni...
AMD at ITC 2014
Leveraging Cross-Operational Test Data for Manufacturing Yield and DPPM/RMA I...
Bosch ConnectedWorld 2017: Striving for Zero DPPM
Use Models for Extending IEEE 1687 to Analog Test
Respond flow chart (rfc)

What's hot (20)

PDF
TEVET-Line-Card
PPTX
Vlsi testing
PDF
6TL NIdays 2010
PDF
Automation of Eddy Current Processes for NDT
PDF
Solution: Long Seam Weld Inspection
PDF
SoftwareAssemblyLineOverview
PDF
Field Pressure Calibration and Equipment Maintenance
PDF
Advances dynamic pile testing technology - Independent Geoscience Pty Ltd
DOC
Controlling interests editors
PDF
GOEPEL Electronics TAP Checker
PDF
Keysight i3070 (antes HP3070) Board Test Interface
PDF
Using Wireless Measurements in Control Applications
PDF
Medalist i3070 08.30p software release
PPTX
FISITA 2018 Keynote Speech by Matthias Popp
 
PDF
Challenges in Assessing Single Event Upset Impact on Processor Systems
PPTX
Physical Design Services
PDF
Optimizing Fire3 and Gas System Design Using the ISA Technical Report ISA TR8...
PPTX
Automation of the Drilling System: What has been done, what is being done, an...
PDF
Case study of dcs upgrade how to reduce stress during execution
PDF
Soft Error Study of ARM SoC at 28 Nanometers
TEVET-Line-Card
Vlsi testing
6TL NIdays 2010
Automation of Eddy Current Processes for NDT
Solution: Long Seam Weld Inspection
SoftwareAssemblyLineOverview
Field Pressure Calibration and Equipment Maintenance
Advances dynamic pile testing technology - Independent Geoscience Pty Ltd
Controlling interests editors
GOEPEL Electronics TAP Checker
Keysight i3070 (antes HP3070) Board Test Interface
Using Wireless Measurements in Control Applications
Medalist i3070 08.30p software release
FISITA 2018 Keynote Speech by Matthias Popp
 
Challenges in Assessing Single Event Upset Impact on Processor Systems
Physical Design Services
Optimizing Fire3 and Gas System Design Using the ISA Technical Report ISA TR8...
Automation of the Drilling System: What has been done, what is being done, an...
Case study of dcs upgrade how to reduce stress during execution
Soft Error Study of ARM SoC at 28 Nanometers
Ad

Similar to Optimal+ GSA 2014 (20)

PPTX
Tug Ot Prez 2010 050510
PPTX
An Industrial Case Study on the Automated Detection of Performance Regression...
PPT
PID Control Of Sampled Measurements - Greg McMillan Deminar Series
PDF
6 sigma LTE release management process improvement
PDF
Deliver Superior Outcomes Using HBT Visualization Tool
PDF
Amar Pradhan: 2013 Sandia National Laboratoies Wind Plant Reliability Workshop
PPTX
Quality assurance of treatment planning system by Rahim Gohar
PPT
Cairo 01 Six Sigma Measure
PPSX
HYPERSIM Relay Protection Webinar
DOC
resume(Cheah Yaw Wah)
PDF
Condition monitoring
PDF
Qualifying a high performance memory subsysten for Functional Safety
PDF
Improvement in Error Resilience in BIST using hamming code
PDF
Improving RoHS Testing with X-ray Fluorescence by Hitachi
PDF
OPAL-RT ePHASORsim Webinar
PDF
智慧檢測技術與工業自動化
PPTX
Semi-automatic Incompatibility Localization for Re-engineered Industrial Soft...
PPTX
case study competition presentation for MBA i
PPT
TPC_Microsoft.ppt
Tug Ot Prez 2010 050510
An Industrial Case Study on the Automated Detection of Performance Regression...
PID Control Of Sampled Measurements - Greg McMillan Deminar Series
6 sigma LTE release management process improvement
Deliver Superior Outcomes Using HBT Visualization Tool
Amar Pradhan: 2013 Sandia National Laboratoies Wind Plant Reliability Workshop
Quality assurance of treatment planning system by Rahim Gohar
Cairo 01 Six Sigma Measure
HYPERSIM Relay Protection Webinar
resume(Cheah Yaw Wah)
Condition monitoring
Qualifying a high performance memory subsysten for Functional Safety
Improvement in Error Resilience in BIST using hamming code
Improving RoHS Testing with X-ray Fluorescence by Hitachi
OPAL-RT ePHASORsim Webinar
智慧檢測技術與工業自動化
Semi-automatic Incompatibility Localization for Re-engineered Industrial Soft...
case study competition presentation for MBA i
TPC_Microsoft.ppt
Ad

Recently uploaded (20)

PDF
A comparative study of natural language inference in Swahili using monolingua...
PDF
Hybrid horned lizard optimization algorithm-aquila optimizer for DC motor
PPTX
MicrosoftCybserSecurityReferenceArchitecture-April-2025.pptx
PDF
Assigned Numbers - 2025 - Bluetooth® Document
PDF
A review of recent deep learning applications in wood surface defect identifi...
PPTX
Chapter 5: Probability Theory and Statistics
PDF
sustainability-14-14877-v2.pddhzftheheeeee
PDF
How ambidextrous entrepreneurial leaders react to the artificial intelligence...
PDF
A contest of sentiment analysis: k-nearest neighbor versus neural network
PDF
Univ-Connecticut-ChatGPT-Presentaion.pdf
PDF
A Late Bloomer's Guide to GenAI: Ethics, Bias, and Effective Prompting - Boha...
PDF
Developing a website for English-speaking practice to English as a foreign la...
DOCX
search engine optimization ppt fir known well about this
PDF
August Patch Tuesday
PDF
CloudStack 4.21: First Look Webinar slides
PDF
Enhancing emotion recognition model for a student engagement use case through...
PPTX
Tartificialntelligence_presentation.pptx
PDF
Taming the Chaos: How to Turn Unstructured Data into Decisions
PDF
Architecture types and enterprise applications.pdf
PPTX
The various Industrial Revolutions .pptx
A comparative study of natural language inference in Swahili using monolingua...
Hybrid horned lizard optimization algorithm-aquila optimizer for DC motor
MicrosoftCybserSecurityReferenceArchitecture-April-2025.pptx
Assigned Numbers - 2025 - Bluetooth® Document
A review of recent deep learning applications in wood surface defect identifi...
Chapter 5: Probability Theory and Statistics
sustainability-14-14877-v2.pddhzftheheeeee
How ambidextrous entrepreneurial leaders react to the artificial intelligence...
A contest of sentiment analysis: k-nearest neighbor versus neural network
Univ-Connecticut-ChatGPT-Presentaion.pdf
A Late Bloomer's Guide to GenAI: Ethics, Bias, and Effective Prompting - Boha...
Developing a website for English-speaking practice to English as a foreign la...
search engine optimization ppt fir known well about this
August Patch Tuesday
CloudStack 4.21: First Look Webinar slides
Enhancing emotion recognition model for a student engagement use case through...
Tartificialntelligence_presentation.pptx
Taming the Chaos: How to Turn Unstructured Data into Decisions
Architecture types and enterprise applications.pdf
The various Industrial Revolutions .pptx

Optimal+ GSA 2014

  • 1. Michael Schuldenfrei, CTO Leveraging Test Data for Quality GSA Quality Team Meeting December, 2014
  • 2. © Optimal+ 2014 2 The Need Shifting from “Defects per Million” to “Defects per Billion”
  • 3. © Optimal+ 2014 3 The Problem No Problem Found 32% Fab Process 28% Test Program 10% Test Operation 4% Test Equipment 26% RMA Source No Problem Found Fab Process Test Program Test Operation Test Equipment
  • 4. © Optimal+ 2014 4 The Challenge BIGDATAEXPERTISE COSTTIME
  • 5. © Optimal+ 2014 5 Big Data – Device DNA ECID ECID ECID ECID ECID WAT WS1 WS2 WAT WS1 WS2 WAT WS1 WS2 WAT WS1 WS2 WS3 WAT WS1 WS2 WS3 FT1 Burn in FT2 Example: One package contains: 5 dice x ~2 WS operations per die x ~1.2 iterations per operation x 3000 parametric measurements + 1000 per-site WAT measurements + 3000 FT measurements A DNA consisting ~35K measurements! An SLT lot with 5000 parts could have 150M historical measurements from hundreds of wafers & FT lots
  • 6. © Optimal+ 2014 6 Back to Basics THE QUALITY QUESTION; IS “GOOD” REALLY GOOD?
  • 7. Outlier Detection Geographic Parametric Escape Prevention Test program issues ATE issues Data Feed Forward (More intelligent decision making) Drift Smart Pairing © Optimal+ 2014 7 Quality Solutions
  • 8. Outlier Detection © Optimal+ 2014 – All rights reserved 8
  • 9. Optimal+ 2014 Company Confidential 9 Outlier Detection – Algorithms D-PAT: Dynamic Part Average Testing NNR: Nearest Neighbor Residual Z-PAT: Z-Axis Part Average Testing GDBN: Good Die in Bad Neighborhood Zonal: Low yield zone-based detection Final Test Post Final-Test operation and Based on Die-ID (ECID etc.) In real-time at Final-Test operation without Die-ID
  • 10. Optimal+ 2014 Company Confidential 10 Cross-Operation Outlier Detection Cross-operational quality based on Die ID Contributing operations ETEST/PCM/WAT Wafer Sort Final-Test Burn-In System Level Test Example: E-Test based bin-switching performed post-Wafer Sort The ability to identify potential bad devices based on E-test data geographical analysis Bin switching occurs post-wafer sort Requires data-feed-forward within the supply chain
  • 11. Escape Prevention © Optimal+ 2014 – All rights reserved 11
  • 12. © Optimal+ 2014 12 Escape Prevention – ATE Freeze A freeze occurs when a tester instrument becomes “stuck” and repeatedly returns the same or similar result for a sequence of parts
  • 13. © Optimal+ 2014 13 Escape Prevention – ATE / TP 13 The STDF “PRR.NUM_TESTS” field tells us the number of tests executed on the part. It should be relatively stable throughout the lot
  • 14. © Optimal+ 2014 14 Escape Prevention – Test Ops Excessive probing – when operation ignores probe mark spec for a device and keeps on probing to get the yield
  • 15. © Optimal+ 2014 15 Escape Prevention – Test Program Human error is one of the main contributors for test escapes and RMA. Here the PE commented a few blocks in the TP for debug and forgot to uncomment before production release: Traditional SBL is design to detect yield issues in which a specific bin count spikes. However human error can result in a drop to 0 which is missed. SBL SBL drop of soft bin 11 from ~3% to 0 following new TP revision
  • 16. © Optimal+ 2014 16 Escape Prevention – Test Program Extremely loose test limits may mask real test performance problems ~95 Sigmas ~95 Sigmas
  • 17. Advanced Quality Solutions © Optimal+ 2014 – All rights reserved 17
  • 18. Implementations: Within the same test area (e.g. WS, FT, etc.) Between test areas (e.g. from WAT to WS to FT) Within a single subcon Between multiple subcons (hub and spoke) Real-time (test program integration) Offline bin-switching Example scenarios: Outlier Detection – drift analysis Pairing – cherry-picking for power & speed combinations Test program tuning SLT / Burn-in reduction © Optimal+ 2014 18 Data Feed Forward
  • 19. © Optimal+ 2014 19 Data Feed Forward – Drift Database at subconTester 1. ECID Data 2. FT1 Measurements Test Program running FT2 operation Real-time data! No test time impact!
  • 20. One or more numeric values representing the perceived quality of a part based on: Wafer geography (e.g. edge vs. center) Outlier detection rule inputs (e.g. GDBN, Z-PAT, D-PAT, etc.) Number of iterations to PASS Overall lot/wafer yield Equipment health during test Parametric test results from multiple operations Etc… © Optimal+ 2014 20 Quality Index Quality Index Lot/Wafer Yield etc. Quality Rule Inputs Wafer Geography
  • 21. © Optimal+ 2014 21 “No Problem Found” Combinations of chips causing issues: IC3 IC2 PCB IC1
  • 22. © Optimal+ 2014 22 Smart Pairing • New methodology to pair IC’s for optimal compatibility • Customer and suppliers agree on recipe for “Best Match” between IC’s (e.g. based on power consumption and speed) • “Quality Index” created based on manufacturing and test data to categorize chips • Data fed-forward to assembly to ensure IC’s pre-sorted into “buckets” based on Quality Index • MCPs and boards are assembled with well-matched components Grade A Grade B Grade C Grade A Grade B Grade C
  • 23. Supreme Quality requires a comprehensive end-to-end approach which takes into account problems arising from: • Equipment • Test Process • Human Error • Material …and much more © Optimal+ 2014 23 Conclusions
  • 24. Q&A Optimal+ 2014 Company Confidential 24