Easing the Reuse of ML Solutions by Interactive Clustering-based Autotuning in Scientific Applications
0 likes8 views
The document discusses a proposed framework for interactive clustering-based autotuning of machine learning (ML) solutions in scientific applications, specifically for biological image segmentation. It highlights challenges in using supervised and unsupervised ML, particularly the need for annotated data and specific tuning for different object types. The framework aims to assist biologists in selecting and tuning ML solutions with minimal user interactions, achieving effective segmentation results while suggesting future improvements in feature space representation.
![Hajiabadi et al. - Easing the Reuse of ML Solutions by Interactive Clustering-based
Autotuning in Scientific Applications
Karlsruhe Institute of Technology - Germany
Related work
Interactive methods
WEKA1
Ilastik2
Drawbacks:
Still need expert knowledge to provide annotations
Mostly focus on cellular segmentation and not subcellular
5
[1] Kanuri N, Abdelkarim AZ, Rathore SA. Trainable WEKA (Waikato Environment for Knowledge Analysis) Segmentation
Tool: Machine-Learning-Enabled Segmentation on Features of Panoramic Radiographs. Cureus. 2022 Jan 31;14(1).
[2] Berg S, Kutra D, Kroeger T, Straehle CN, Kausler BX, Haubold C, Schiegg M, Ales J, Beier T, Rudy M, Eren K.
Ilastik: interactive machine learning for (bio) image analysis. Nature Methods. 2019 Dec;16(12):1226-32.
8/27/2022
WEKA /
Ilastik](https://image.slidesharecdn.com/daideeasingthereuseofmlsolutionsbyinteractiveclustering-basedautotuninginscientificapplications-220831084732-197b7f8e/85/Easing-the-Reuse-of-ML-Solutions-by-Interactive-Clustering-based-Autotuning-in-Scientific-Applications-5-320.jpg)
![Hajiabadi et al. - Easing the Reuse of ML Solutions by Interactive Clustering-based
Autotuning in Scientific Applications
Karlsruhe Institute of Technology - Germany
We used input images recorded by STEDD microscopy in a previous
study [1, 2].
The images are cell nuclei in fixed zebrafish embryos, where RNA was
labeled.
8/27/2022
8
Case study
[1] Pancholi et al., “Rna polymerase ii clusters form in line with surface condensation on regulatory chromatin,” Molecular systems biology, vol.
17, no. 9, p. e10272, 2021
[2] L. Hilbert, “Analysis of RNA polymerase II phosphorylation in STimulated Emission Double Depletion (STEDD) microscopy images,” Jun.
2021.](https://image.slidesharecdn.com/daideeasingthereuseofmlsolutionsbyinteractiveclustering-basedautotuninginscientificapplications-220831084732-197b7f8e/85/Easing-the-Reuse-of-ML-Solutions-by-Interactive-Clustering-based-Autotuning-in-Scientific-Applications-8-320.jpg)
Easing the Reuse of ML Solutions by Interactive Clustering-based Autotuning in Scientific Applications
- 1. KIT – The Research University in the Helmholtz Association www.kit.edu Easing the Reuse of ML Solutions by Interactive Clustering-based Autotuning in Scientific Applications Hamideh Hajiabadi, Lennart Hilbert, Anne Koziolek Karlsruhe Institute of Technology, Karlsruhe, Germany Institute of Information Security and Dependability Institute for Biological and Chemical Systems
- 2. Hajiabadi et al. - Easing the Reuse of ML Solutions by Interactive Clustering-based Autotuning in Scientific Applications Karlsruhe Institute of Technology - Germany Difficulties of using ML in scientific applications 2 8/27/2022
- 3. Hajiabadi et al. - Easing the Reuse of ML Solutions by Interactive Clustering-based Autotuning in Scientific Applications Karlsruhe Institute of Technology - Germany Challenges of supervised vs unsupervised ML reuse 3 8/27/2022 • automatically tunning ☺ • annotations by experienced domain specialists • sufficiently comprehensive annotated data to avoid over-fitting • re-train on new data supervised unsupervised • manually tunning • no annotations is required ☺ • re-tune on new data
- 4. Hajiabadi et al. - Easing the Reuse of ML Solutions by Interactive Clustering-based Autotuning in Scientific Applications Karlsruhe Institute of Technology - Germany 1. Supervised segmentation need annotated data and it is very expensive. 2. The segmentation algorithm cannot be used out of the box. It indeed needs tuning 3. Objects are in different size, intensity, shape and sometimes with unclear boundary 4. Different settings needs to apply in each object type 8/27/2022 4 Scientific use case: biological image segmentation RNA distribution in fixed zebrafish embryo
- 5. Hajiabadi et al. - Easing the Reuse of ML Solutions by Interactive Clustering-based Autotuning in Scientific Applications Karlsruhe Institute of Technology - Germany Related work Interactive methods WEKA1 Ilastik2 Drawbacks: Still need expert knowledge to provide annotations Mostly focus on cellular segmentation and not subcellular 5 [1] Kanuri N, Abdelkarim AZ, Rathore SA. Trainable WEKA (Waikato Environment for Knowledge Analysis) Segmentation Tool: Machine-Learning-Enabled Segmentation on Features of Panoramic Radiographs. Cureus. 2022 Jan 31;14(1). [2] Berg S, Kutra D, Kroeger T, Straehle CN, Kausler BX, Haubold C, Schiegg M, Ales J, Beier T, Rudy M, Eren K. Ilastik: interactive machine learning for (bio) image analysis. Nature Methods. 2019 Dec;16(12):1226-32. 8/27/2022 WEKA / Ilastik
- 6. Hajiabadi et al. - Easing the Reuse of ML Solutions by Interactive Clustering-based Autotuning in Scientific Applications Karlsruhe Institute of Technology - Germany We propose: A combination of auto-tuning and active learning In scenarios where no training data and no objective function are available But a metric approximating the quality can be defined 8/27/2022 6 Proposed approach Proposed framework Benefit: It helps biologists to choose the ML solution and tune the parameters with few interactions
- 7. Hajiabadi et al. - Easing the Reuse of ML Solutions by Interactive Clustering-based Autotuning in Scientific Applications Karlsruhe Institute of Technology - Germany The proposed framework 7 8/27/2022
- 8. Hajiabadi et al. - Easing the Reuse of ML Solutions by Interactive Clustering-based Autotuning in Scientific Applications Karlsruhe Institute of Technology - Germany We used input images recorded by STEDD microscopy in a previous study [1, 2]. The images are cell nuclei in fixed zebrafish embryos, where RNA was labeled. 8/27/2022 8 Case study [1] Pancholi et al., “Rna polymerase ii clusters form in line with surface condensation on regulatory chromatin,” Molecular systems biology, vol. 17, no. 9, p. e10272, 2021 [2] L. Hilbert, “Analysis of RNA polymerase II phosphorylation in STimulated Emission Double Depletion (STEDD) microscopy images,” Jun. 2021.
- 9. Hajiabadi et al. - Easing the Reuse of ML Solutions by Interactive Clustering-based Autotuning in Scientific Applications Karlsruhe Institute of Technology - Germany 9 First step: object detection Framework detects the objects and prepares the object set of size 60 × 60 pixels 8/27/2022
- 10. Hajiabadi et al. - Easing the Reuse of ML Solutions by Interactive Clustering-based Autotuning in Scientific Applications Karlsruhe Institute of Technology - Germany The frameworks clusters the objects. Number of clusters is given to the framework 8/27/2022 10 Second step: object set clustering
- 11. Hajiabadi et al. - Easing the Reuse of ML Solutions by Interactive Clustering-based Autotuning in Scientific Applications Karlsruhe Institute of Technology - Germany In each interaction 4 images based on the defined metric are selected The results of four segmentation algorithm is then showed Via user interaction, user selects the best algorithm The selected augmentation method is then adjusted in the same manner. 8/27/2022 11 Third step: framework suggests some segmentation
- 12. Hajiabadi et al. - Easing the Reuse of ML Solutions by Interactive Clustering-based Autotuning in Scientific Applications Karlsruhe Institute of Technology - Germany RQ1: How effective is our framework in improving the quality of segmentation? RQ2: How much interactions the framework needs from users to reuse the ML solutions? 8/27/2022 12 Evaluation
- 13. Hajiabadi et al. - Easing the Reuse of ML Solutions by Interactive Clustering-based Autotuning in Scientific Applications Karlsruhe Institute of Technology - Germany Evaluation process 13 8/27/2022
- 14. Hajiabadi et al. - Easing the Reuse of ML Solutions by Interactive Clustering-based Autotuning in Scientific Applications Karlsruhe Institute of Technology - Germany A sample questionnaire provided to expert II 14 8/27/2022
- 15. Hajiabadi et al. - Easing the Reuse of ML Solutions by Interactive Clustering-based Autotuning in Scientific Applications Karlsruhe Institute of Technology - Germany Results 15 8/27/2022 RQ1: How effective is our framework in improving the quality of segmentation? Framework Choice Percentage Results obtained by our framework outperforms 80% Results obtained after expert tuning outperforms 10% Both results look almost similar 10% It depends on …. 0%
- 16. Hajiabadi et al. - Easing the Reuse of ML Solutions by Interactive Clustering-based Autotuning in Scientific Applications Karlsruhe Institute of Technology - Germany 8/27/2022 16 Results RQ2: How much interactions the framework needs from users to reuse the ML solutions? Framework Total number of intraction Spent time Simplicity 1: not simple 2: maximally simple User 1 8 6 minutes 6 User 2 10 7 minutes 6 User 3 6 5 minutes 7 Average 8 6 minutes 6
- 17. Hajiabadi et al. - Easing the Reuse of ML Solutions by Interactive Clustering-based Autotuning in Scientific Applications Karlsruhe Institute of Technology - Germany Only example data from one type of biological sample, recorded on one type of microscope Three users working in the same laboratory tested the framework by tuning segmentation on the object set. Only one expert is used to visually assess the quality of segmentation results 8/27/2022 17 Threats to validity
- 18. Hajiabadi et al. - Easing the Reuse of ML Solutions by Interactive Clustering-based Autotuning in Scientific Applications Karlsruhe Institute of Technology - Germany Summary and future work 18 8/27/2022 • combination of autotuning and active learning • object-type specific adjustment • a few user interactions Future work: improving the feature space representation of objects by interactive fashion