The Green Lab - [07-B] Hypothesis Testing
0 likes1,764 views
The document discusses hypotheses in experimentation, specifically the null hypothesis (H0) which suggests no real trends, and the alternative hypothesis (Ha) which indicates real trends. It emphasizes the importance of verifying the null hypothesis and rejecting it to draw conclusions, citing Popper's philosophy on scientific statements. Various resources and examples related to hypothesis testing and effect sizes in software engineering are mentioned.
![The Green Lab - [07-B] Hypothesis Testing](https://image.slidesharecdn.com/07-bhypothesistesting-161031134601/85/The-Green-Lab-07-B-Hypothesis-Testing-1-320.jpg)
The Green Lab - [07-B] Hypothesis Testing
- 2. Vrije Universiteit Amsterdam Hypothesis: a formal statement about a phenomenon ● Null hypothesis H0 : no real trends or patterns in the experiment setting (aka ~Q) ● Alternative hypothesis Ha : there are real trends or patterns in the experiment setting (aka Q)
- 3. Vrije Universiteit Amsterdam ● We aim at verifying the null hypothesis (~Q) ○ we test the null hypothesis H0 We can reject the null hypothesis → we can draw conclusions This comes from Popper (1959): any statement in a scientific field is true until anybody can contradict it ● Aiming at verifying Q is WRONG ○ Provides no insight on the conjecture
- 6. Vrije Universiteit Amsterdam ● ○ "P-value in statistical significance testing" by User:Repapetilto @ Wikipedia & User:Chen-Pan Liao @ Wikipedia - https://en.wikipedia.org/wiki/File:P_value.png. Licensed under CC BY-SA 3.0 via Wikimedia Commons
- 8. Vrije Universiteit Amsterdam ● ○ ○ α ● ○ ○ β
- 10. Vrije Universiteit Amsterdam ● ○ β ●
- 12. Vrije Universiteit Amsterdam ● α ○ α ○ ● β ○ ○ ●
- 16. Vrije Universiteit Amsterdam α ● α ○ ● 0.38 α
- 20. Vrije Universiteit Amsterdam ● Analysis and Interpretation: Chapter 10 Experimentation in SE, Wohlin et al. (Blackboard) ● Hypothesis Testing: Chapter 10, Section 3 Experimentation in SE, Wohlin et al. (Blackboard) ● "It's the effect size, stupid" (Coe, R.) ● Plenty of online resourcestutorials
- 21. Vrije Universiteit Amsterdam ● Coin toss example and other content from Empirical Methods in Software Engineering, Marco Torchiano, Politecnico di Torino - http://softeng.polito.it/EMSE/ ● Effect Size from Basics of Empirical Software Engineering, Massimiliano Di Penta, University of Sannio, Italy