Ruud Cox - Testing in a Medical Device Context - EuroSTAR 2012
1 like629 views
The document discusses the challenges and best practices of testing in the medical device sector, emphasizing the need for structured approaches like verification and validation to ensure compliance with regulations. It highlights issues such as incomplete testing, documentation, and the lack of agile practices within teams, suggesting that exploratory testing and feedback loops can enhance product quality. Furthermore, it stresses the importance of traceability and independent testing to provide objective evidence of compliance and performance.
![Component Testing
Software Interface
Testability
Exploratory Testing
Coverage/risk outline
Scripted Testing
[Test]
[RequirementTags("Req1", "Req7")]
void FooBar()
{
Assert.That(…);
}
Unit tests annotated with requirement tags for
traceability
Specify test cases for high risk specified requirements only](https://image.slidesharecdn.com/th5-testing-in-a-medical-device-context-1-141029105940-conversion-gate02/85/Ruud-Cox-Testing-in-a-Medical-Device-Context-EuroSTAR-2012-11-320.jpg)
Ruud Cox - Testing in a Medical Device Context - EuroSTAR 2012
- 1. Testing in a Medical Device Context Limitations are few Ruud Cox, Improve Quality Services www.eurostarconferences.com @esconfs #esconfs
- 2. From Research to Manufacturing Hospital Market Segment Healthcare, Deep Brain Stimulation Big Organisation Divisions Manufacturer Research Law Regulations Products Probe Tools to support Surgical Procedures Product Prototypes Hospital Hospital Hospital New organisation Manufacturer
- 3. Deep Brain Stimulation A pacemaker for the brain
- 4. Almost no testing Incomplete, overdue documentation
- 5. Organise Testing Approach: (Verification & Validation) Exploratory Testing Learning, investigation, discovery followed by Scripted Testing to demonstrate that the product conforms to specified requirements and to provide objective evidence for submission. Conformance, Compliance
- 6. Exploratory Testing In the given situation, this was the best way to • Learn more about the product • Add value by providing feedback Charters, Debriefs, Screen Recordings, Managed A big success
- 7. Regulations and Testing Verification only §820.3(aa) Verification means confirmation by examination and provision of objective evidence that specified requirements have been fulfilled. Testing is one of many possible verification activities DESIGN CONTROL GUIDANCE FOR MEDICAL DEVICE MANUFACTURERS This Guidance relates to FDA 21 CFR 820.30 and Sub-clause 4.4 of ISO 9001
- 8. Objective Evidence Check pressure. Check that the pressure is OK. Record the pressure here ___. Check that the pressure is between 50 and 75 psi. A check requires a decision rule i.e. specific, measurable requirements Example from Medical Device Software Verification, Validation and Compliance by David A. Vogel ISBN 1596934220
- 9. Specified Requirements • User Stories – Sometimes with acceptance criteria • Traditional requirements – “The product shall...” • User Stories and traditional requirements were not linked A real challenge!
- 10. System Global Design Component Graphical User Interface Business Logic Component Component Component
- 11. Component Testing Software Interface Testability Exploratory Testing Coverage/risk outline Scripted Testing [Test] [RequirementTags("Req1", "Req7")] void FooBar() { Assert.That(…); } Unit tests annotated with requirement tags for traceability Specify test cases for high risk specified requirements only
- 13. Structure Functions Data Time Operations Platform Coverage Bugs Issues Risks
- 14. Structure Functions Data Bugs Issues Risks Time Operations Platform Coverage/Risk outline
- 15. Why PASS and FAIL is not enough PASS only means that no problem was found, which doesn’t mean that there’s no problem Comparable Products • Lots of remarks during evaluation at hospital • Our product side-by-side to competitors product
- 16. Independent Testing • Testers were reponsible for – ALL test documentation – Providing Objective Evidence • Developers did some unit testing but – Their tests were undocumented – Could not be used as Objective Evidence Why does testing take so long?
- 17. Time-out JIGGLE! Testing Incomplete, overdue specified requirements. Independent testing. “I don’t know nothing about testing.” What does the team know about or does have experience with working in a Medical Device Context? Agile/Scrum Almost no Agile practices. The team didn’t live Scrum. What does the team know about or does have experience with Agile/Scrum?
- 18. Regulations and Agile/Scrum CONCURRENT ENGINEERING. Although the waterfall model is a useful tool for introducing design controls, its usefulness in practice is limited. DESIGN CONTROL GUIDANCE FOR MEDICAL DEVICE MANUFACTURERS This Guidance relates to FDA 21 CFR 820.30 and Sub-clause 4.4 of ISO 9001 Agile TIR SW Committee Draft 1.0 Guidance on the use of AGILE practices in the development of medical device software
- 19. Improvements after Time-out Testing • IEC 62304 training, Less independent testing. Agile/Scrum • Product Backlog, Sprint Planning, Definition of Done, Scrum Board, Sprint Review, Retrospectives, Smaller Teams each with their own Scrum Master • Scrum Master, Product Owner training • Agile/Scrum Coach Overdue specified requirements
- 20. Other Identified Risks Component not traceable to global design and risk analysis 3D mathematical model was not specified Claim these results
- 21. Clarify mission System Architect Status reporting Director Quality & Regulations Program Director Test Architect Product Owner
- 22. Observe the work of testing A diary might help
- 23. Lessons Learned, Conclusions • A feedback loop is mandatory – Mission – Status Reporting • Medical Device Context is not limiting – Exploratory testing is possible – Agile/Scrum is possible No matter what the problem is, it’s always a people problem. Gerald M. Weinberg
- 24. Questions Ruud Cox email rco@improveqs.nl Twitter @ruudcox Blog http://ruudcox.wordpress.com/