Elements of Applied Mathematics for Engineers

Draft
The free 5,000 pages super-quick, super-painless undergraduate transportable Book
on Elementary Applied Mathematics for Engineers (EAME)
Sciences.chISOZ Vincent distribution
3rd Edition
{ISBN 978-2-8399-0932-7} October 2014
Compiled with LATEX2εon TEXmaker
EAME-3 work is licensed under a Creative Commons "Attribution 3.0 Unported" license.
paternity; no commercial usage; sharing of conditions identical
https://creativecommons.org/licenses/by-nc-sa/4.0/
2001-2015: One book to rule them (almost) all!

Draft
SCIENTIFIC EVOLUTION SÀRL
EAME
3RD EDITION
The Isoz Compendium on
Elementary Applied Mathematics for Engineers
Redactor:
Vincent ISOZ
isoz@sciences.ch
Supervisor:
Vincent ISOZ
2016–05–15

Draft
Revision History
Revision Date Author(s) Description
3.1 2014-10-01 VI French 3rd Edition translation into English (only
5,000 ﬁrst pages freely available!). Translation
progress ∼ 60%
iii

Draft
Contents
1 Warnings 1
2 Acknowledgements 20
3 Introduction 22
4 Arithmetic 42
5 Algebra 675
6 Analysis 1148
7 Geometry 1307
8 Mechanics 1618
9 Electromagnetism 1763
10 Atomistic 1884
11 Cosmology 1957
12 Chemistry 2049
13 Theoretical Computing 2106
14 Social Sciences 2302
15 Engineering 2617
16 Epilogue 2777
17 Biographies 2779
18 Chronology 2872
v

Draft
19 Humour 2902
20 Links 2980
21 Quotes 2990
22 Change Log 2995
23 Nomenclature 3012
24 About the Redactor 3014
List of Figures 3018
List of Tables 3038
Bibliography 3042
Index 3046
vi

Draft
Table of Contents
1 Warnings 1
1.1 Impressum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1.1 Use of content . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1.2 How to use this book . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1.2.1 Companion eBooks, Forum and Quiz/Flashcards . . . . . . . 3
1.1.3 Data Protection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.1.4 Use of data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.1.5 Data transmission . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.1.6 Agreement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2 License . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.2.1 Preamble . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.2.2 Applicability and Deﬁnitions . . . . . . . . . . . . . . . . . . . . . . . 8
1.2.3 Verbatim Copying . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.2.4 Copying in Quantity . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.2.5 Modiﬁcations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.2.6 Combining Documents . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.2.7 Collections of Documents . . . . . . . . . . . . . . . . . . . . . . . . 12
1.2.8 Aggregation with independant Works . . . . . . . . . . . . . . . . . . 12
1.2.9 Translation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.2.10 Termination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.2.11 Future revisions of this License . . . . . . . . . . . . . . . . . . . . . . 13
1.3 Only in non-free release . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2 Acknowledgements 20
3 Introduction 22
3.1 Forewords . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.2 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.2.1 Descartes’ Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.2.2 Archimedean Oath . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.3 Vocabulary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.3.1 On Sciences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.3.2 Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3.4 Science and Faith . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
4 Arithmetic 42
4.1 Proof Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
4.1.0.1 Foundations Crisis . . . . . . . . . . . . . . . . . . . . . . . 45
4.1.1 Paradoxes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
vii

Draft
4.1.1.1 Hypothetical-Deductive Reasoning . . . . . . . . . . . . . . 51
4.1.2 Propositional Calculus . . . . . . . . . . . . . . . . . . . . . . . . . . 52
4.1.2.1 Propositions . . . . . . . . . . . . . . . . . . . . . . . . . . 53
4.1.2.2 Connectors . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
4.1.2.3 Decision procedures . . . . . . . . . . . . . . . . . . . . . . 61
4.1.2.3.1 Non-axiomatic procedural decisions . . . . . . . . . 62
4.1.2.3.2 Axiomatic procedural decisions . . . . . . . . . . . 62
4.1.2.4 Quantiﬁers . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.1.3 Predicate Calculus . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
4.1.3.1 Grammar . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
4.1.3.2 Language . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
4.1.3.2.1 Symbols . . . . . . . . . . . . . . . . . . . . . . . 69
4.1.3.2.2 Terms . . . . . . . . . . . . . . . . . . . . . . . . . 70
4.1.3.2.3 Formulas . . . . . . . . . . . . . . . . . . . . . . . 72
4.2 Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
4.2.1 Digital Bases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
4.2.2 Type of Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
4.2.2.1 Natural Integer Numbers . . . . . . . . . . . . . . . . . . . . 84
4.2.2.1.1 Peano axioms . . . . . . . . . . . . . . . . . . . . 86
4.2.2.1.2 Odd, Even and Perfect Numbers . . . . . . . . . . . 87
4.2.2.1.3 Prime Numbers . . . . . . . . . . . . . . . . . . . . 87
4.2.2.2 Relative Integer Numbers . . . . . . . . . . . . . . . . . . . 89
4.2.2.3 Rational Numbers . . . . . . . . . . . . . . . . . . . . . . . 90
4.2.2.4 Irrational Numbers . . . . . . . . . . . . . . . . . . . . . . . 93
4.2.2.5 Real Numbers . . . . . . . . . . . . . . . . . . . . . . . . . 95
4.2.2.6 Transﬁnite Numbers . . . . . . . . . . . . . . . . . . . . . . 97
4.2.2.7 Complex Numbers . . . . . . . . . . . . . . . . . . . . . . . 100
4.2.2.7.1 Geometric Interpretation of Complex Numbers . . . 105
4.2.2.7.1.1 Fresnel Vectors (phasors) . . . . . . . . . . 109
4.2.2.7.2 Transformation in the plane . . . . . . . . . . . . . 110
4.2.2.8 Quaternion Numbers . . . . . . . . . . . . . . . . . . . . . . 115
4.2.2.8.1 Matrix Interpretation of Quaternions . . . . . . . . 121
4.2.2.8.2 Rotations with Quaternions . . . . . . . . . . . . . 122
4.2.2.9 Algebraic and Transcendental Numbers . . . . . . . . . . . . 128
4.2.2.10 Abstract Numbers (variables) . . . . . . . . . . . . . . . . . 131
4.2.2.10.1 Domain of a Variable . . . . . . . . . . . . . . . . 132
4.3 Arithmetic Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
4.3.1 Binary Relations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
4.3.1.1 Equalities . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
4.3.1.2 Comparators . . . . . . . . . . . . . . . . . . . . . . . . . . 137
4.3.2 Fundamental Arithmetic Laws . . . . . . . . . . . . . . . . . . . . . . 144
4.3.2.1 Addition . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
4.3.2.2 Subtraction . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
4.3.2.3 Multiplication . . . . . . . . . . . . . . . . . . . . . . . . . 151
4.3.2.4 Division . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
4.3.2.4.1 n-root . . . . . . . . . . . . . . . . . . . . . . . . . 157
4.3.3 Arithmetic Polynomials . . . . . . . . . . . . . . . . . . . . . . . . . . 160
4.3.4 Absolute Value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
viii

Draft
4.3.5 Calculation Rules (operators priorities) . . . . . . . . . . . . . . . . . . 164
4.4 Number Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
4.4.1 Principle of good order . . . . . . . . . . . . . . . . . . . . . . . . . . 169
4.4.2 Induction Principle . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
4.4.3 Divisibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
4.4.3.1 Euclidean Division . . . . . . . . . . . . . . . . . . . . . . . 174
4.4.3.1.1 Greatest common divisor . . . . . . . . . . . . . . 176
4.4.3.2 Euclidean Algorithm . . . . . . . . . . . . . . . . . . . . . . 179
4.4.3.3 Least Common Multiple . . . . . . . . . . . . . . . . . . . . 183
4.4.3.4 Fundamental Theorem of Arithmetic . . . . . . . . . . . . . 186
4.5 Set Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
4.5.1 Zermelo-Fraenkel Axiomatic . . . . . . . . . . . . . . . . . . . . . . . 198
4.5.1.1 Cardinals . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
4.5.1.2 Cartesian Product . . . . . . . . . . . . . . . . . . . . . . . 206
4.5.1.3 Intervals . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
4.5.2 Set Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
4.5.2.1 Inclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
4.5.2.2 Intersection . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
4.5.2.3 Union . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210
4.5.2.4 Difference . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
4.5.2.5 Symmetric Difference . . . . . . . . . . . . . . . . . . . . . 213
4.5.2.6 Product . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214
4.5.2.7 Complementarity . . . . . . . . . . . . . . . . . . . . . . . . 215
4.5.3 Functions and Applications . . . . . . . . . . . . . . . . . . . . . . . . 217
4.5.3.1 Cantor-Bernstein Theorem . . . . . . . . . . . . . . . . . . . 224
4.6 Probabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233
4.6.1 Event Universe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
4.6.2 Kolmogorov’s Axioms . . . . . . . . . . . . . . . . . . . . . . . . . . 235
4.6.3 Conditional Probabilities . . . . . . . . . . . . . . . . . . . . . . . . . 241
4.6.3.1 Conditional Expectation . . . . . . . . . . . . . . . . . . . . 246
4.6.3.2 Bayesian Networks . . . . . . . . . . . . . . . . . . . . . . . 248
4.6.4 Martingales . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260
4.6.5 Combinatorial Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . 262
4.6.5.1 Simple Arrangements with Repetition . . . . . . . . . . . . . 263
4.6.5.2 Simple Permutations without Repetitions . . . . . . . . . . . 264
4.6.5.3 Simple Permutations with Repetitions . . . . . . . . . . . . . 265
4.6.5.4 Simple Arrangements with Repetitions . . . . . . . . . . . . 266
4.6.5.5 Simple Combinations without Repetitions . . . . . . . . . . . 267
4.6.5.6 Simple Combinations with Repetitions . . . . . . . . . . . . 269
4.6.6 Markov Chains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
4.7 Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276
4.7.1 Samples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278
4.7.2 Averages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
4.7.2.1 Laplace Smoothing . . . . . . . . . . . . . . . . . . . . . . . 295
4.7.2.2 Means and Averages properties . . . . . . . . . . . . . . . . 296
4.7.3 Type of variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301
4.7.3.1 Discrete Variables and Moments . . . . . . . . . . . . . . . . 302
4.7.3.1.1 Mean and Deviation of Discrete Random Variables . 303
ix

Draft
4.7.3.1.2 Discrete Covariance . . . . . . . . . . . . . . . . . 313
4.7.3.1.2.1 Anscombe’s famous quartet . . . . . . . . . 319
4.7.3.1.3 Mean and Variance of the Average . . . . . . . . . 321
4.7.3.1.4 Coefﬁcient of Correlation . . . . . . . . . . . . . . 322
4.7.3.2 Continuous Variables and Moments . . . . . . . . . . . . . . 327
4.7.4 Fundamental postulate of statistics . . . . . . . . . . . . . . . . . . . . 329
4.7.5 Diversity Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329
4.7.6 Distribution Functions (probabilities laws) . . . . . . . . . . . . . . . . 331
4.7.6.1 Discrete Uniform Distribution . . . . . . . . . . . . . . . . . 333
4.7.6.2 Bernoulli Distribution . . . . . . . . . . . . . . . . . . . . . 336
4.7.6.3 Geometric Distribution . . . . . . . . . . . . . . . . . . . . . 337
4.7.6.4 Binomial Distribution . . . . . . . . . . . . . . . . . . . . . 341
4.7.6.5 Negative Binomial Distribution . . . . . . . . . . . . . . . . 349
4.7.6.6 Hypergeometric Distribution . . . . . . . . . . . . . . . . . . 354
4.7.6.7 Multinomial Distribution . . . . . . . . . . . . . . . . . . . . 360
4.7.6.8 Poisson Distribution . . . . . . . . . . . . . . . . . . . . . . 364
4.7.6.9 Normal & Gauss-Laplace Distribution . . . . . . . . . . . . . 367
4.7.6.9.1 Sum of two random Normal variables . . . . . . . . 375
4.7.6.9.2 Product of two random Normal variables . . . . . . 377
4.7.6.9.3 Bivariate Normal Distribution . . . . . . . . . . . . 379
4.7.6.9.4 Normal Reduced Centered Distribution . . . . . . . 384
4.7.6.9.5 Henry’s Line . . . . . . . . . . . . . . . . . . . . . 385
4.7.6.9.6 Q-Q plot . . . . . . . . . . . . . . . . . . . . . . . 389
4.7.6.10 Log-Normal Distribution . . . . . . . . . . . . . . . . . . . . 391
4.7.6.11 Continuous Uniform Distribution . . . . . . . . . . . . . . . 394
4.7.6.12 Triangular Distribution . . . . . . . . . . . . . . . . . . . . . 398
4.7.6.13 Pareto Distribution . . . . . . . . . . . . . . . . . . . . . . . 400
4.7.6.14 Exponential Distribution . . . . . . . . . . . . . . . . . . . . 406
4.7.6.15 Cauchy Distribution . . . . . . . . . . . . . . . . . . . . . . 408
4.7.6.16 Beta Distribution . . . . . . . . . . . . . . . . . . . . . . . . 411
4.7.6.17 Gamma Distribution . . . . . . . . . . . . . . . . . . . . . . 414
4.7.6.18 Chi-Square (pearson) Distribution . . . . . . . . . . . . . . . 419
4.7.6.19 Student Distribution . . . . . . . . . . . . . . . . . . . . . . 423
4.7.6.20 Fisher Distribution . . . . . . . . . . . . . . . . . . . . . . . 427
4.7.6.21 Benford Distribution . . . . . . . . . . . . . . . . . . . . . . 430
4.7.7 Likelihood Estimators . . . . . . . . . . . . . . . . . . . . . . . . . . 433
4.7.7.1 Normal Distribution MLE . . . . . . . . . . . . . . . . . . . 435
4.7.7.2 Poisson Distribution MLE . . . . . . . . . . . . . . . . . . . 439
4.7.7.3 Binomial (and Geometric) Distribution MLE . . . . . . . . . 440
4.7.7.4 Weibull Distribution MLE . . . . . . . . . . . . . . . . . . . 441
4.7.7.5 Gamma Distribution MLE . . . . . . . . . . . . . . . . . . . 444
4.7.8 Finite Population Correction Factor . . . . . . . . . . . . . . . . . . . 445
4.7.9 Conﬁdence Intervals . . . . . . . . . . . . . . . . . . . . . . . . . . . 448
4.7.9.1 C.I. on the Mean with known Variance . . . . . . . . . . . . 450
4.7.9.2 C.I. on the Variance with known Mean . . . . . . . . . . . . 454
4.7.9.3 C.I. on the Variance with empirical Mean . . . . . . . . . . . 458
4.7.9.4 C.I. on the Mean with known unbiased Variance . . . . . . . 459
4.7.9.5 Binomial exact Test . . . . . . . . . . . . . . . . . . . . . . 463
x

Draft
4.7.9.6 C.I. for a Proportion . . . . . . . . . . . . . . . . . . . . . . 467
4.7.9.6.1 Test of equality of two Proportions . . . . . . . . . 471
4.7.9.7 Sign Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . 473
4.7.9.8 Mood’s Median Test . . . . . . . . . . . . . . . . . . . . . . 475
4.7.9.9 Poisson Test (1 sample) . . . . . . . . . . . . . . . . . . . . 477
4.7.9.10 Poisson Test (2 samples) . . . . . . . . . . . . . . . . . . . . 480
4.7.9.11 Confidence/Tolerance/Prediction Interval . . . . . . . . . . . 483
4.7.10 Weak Law of Large Numbers . . . . . . . . . . . . . . . . . . . . . . . 485
4.7.11 Characteristic Function . . . . . . . . . . . . . . . . . . . . . . . . . . 489
4.7.12 Central Limit Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . 493
4.7.13 Univariate Hypothesis and Adequation tests . . . . . . . . . . . . . . . 499
4.7.13.1 Direction of hypothesis test . . . . . . . . . . . . . . . . . . 502
4.7.13.1.1 Simpson’s Paradox (sophism) . . . . . . . . . . . . 508
4.7.13.2 Power of a test . . . . . . . . . . . . . . . . . . . . . . . . . 510
4.7.13.3 Power of the one sample Z-test . . . . . . . . . . . . . . . . 512
4.7.13.4 Power of the one and two samples P-test . . . . . . . . . . . 514
4.7.13.5 Analysis of Variance with one fixed factor . . . . . . . . . . . 516
4.7.13.6 Analysis of Variance with two fixed factors without repetitions 529
4.7.13.7 Analysis of Variance with two fixed factors with repetitions . 545
4.7.13.8 Multifactor ANOVA with Repeated measures . . . . . . . . . 551
4.7.13.9 Greaco Latin Square ANOVA . . . . . . . . . . . . . . . . . 552
4.7.13.10 Cochran C test . . . . . . . . . . . . . . . . . . . . . . . . . 559
4.7.13.11 Adequation Tests (goodness of fit tests) . . . . . . . . . . . . 561
4.7.13.11.1 Pearson’s chi-squared GoF test . . . . . . . . . . . 561
4.7.13.11.2 Kolmogorov-Smirnov GoF test . . . . . . . . . . . 566
4.7.13.11.3 Ryan-Joiner GoF test . . . . . . . . . . . . . . . . . 572
4.7.13.11.4 Anderson-Darling GoF test . . . . . . . . . . . . . 576
4.7.14 Robustness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 586
4.7.14.1 Rank Statistics . . . . . . . . . . . . . . . . . . . . . . . . . 587
4.7.14.1.1 L-Statistics . . . . . . . . . . . . . . . . . . . . . . 587
4.7.14.1.2 Ranks Distribution Law . . . . . . . . . . . . . . . 588
4.7.14.1.3 Wilcoxon Rank Sum Test . . . . . . . . . . . . . . 589
4.7.14.1.4 Mann-Witheny Rank Sum Test . . . . . . . . . . . 598
4.7.14.1.5 Treatment of equalities . . . . . . . . . . . . . . . . 604
4.7.14.1.6 One sample Wilcoxon rank sum signed test . . . . . 606
4.7.14.1.7 Wilcoxon rank sum signed test for two paired samples610
4.7.14.1.8 Kruskal-Wallis test . . . . . . . . . . . . . . . . . . 612
4.7.14.1.9 Friedman Test . . . . . . . . . . . . . . . . . . . . 616
4.7.14.1.10 Spearman Rank Correlation Coefficient . . . . . . . 619
4.7.14.2 Range Statistics . . . . . . . . . . . . . . . . . . . . . . . . 622
4.7.14.2.1 Tukey’s Range Test . . . . . . . . . . . . . . . . . 625
4.7.14.3 Extreme Value Theory . . . . . . . . . . . . . . . . . . . . . 629
4.7.15 Multivariate Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . 630
4.7.15.1 Principal Component Analysis . . . . . . . . . . . . . . . . . 631
4.7.15.2 Correspondence Factorial Analysis (AFC) . . . . . . . . . . 647
4.7.15.3 Chi-2 Test of Independence . . . . . . . . . . . . . . . . . . 652
4.7.15.4 Cramér’s V . . . . . . . . . . . . . . . . . . . . . . . . . . . 656
4.7.15.5 Exact Fisher Test . . . . . . . . . . . . . . . . . . . . . . . . 660
xi

Draft
4.7.15.6 Cohen’s kappa agreement . . . . . . . . . . . . . . . . . . . 665
4.7.16 Propagation of Errors (experimental uncertainty analysis) . . . . . . . . 667
4.7.16.1 Absolute and Relative Uncertainties (Direct calculation of bias)667
4.7.16.2 Statistical Errors . . . . . . . . . . . . . . . . . . . . . . . . 668
4.7.16.3 Repeatability . . . . . . . . . . . . . . . . . . . . . . . . . . 670
4.7.16.4 Error propagation (linearized approximation) . . . . . . . . . 671
4.7.16.5 Significative Numbers . . . . . . . . . . . . . . . . . . . . . 673
5 Algebra 675
5.1 Calculus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 677
5.1.1 Equations and Inequations . . . . . . . . . . . . . . . . . . . . . . . . 678
5.1.1.1 Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 679
5.1.1.2 Inequations . . . . . . . . . . . . . . . . . . . . . . . . . . . 683
5.1.2 Remarkable Identities . . . . . . . . . . . . . . . . . . . . . . . . . . . 687
5.1.3 Polynomials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 691
5.1.3.1 Euclidean Divison of Polynomials . . . . . . . . . . . . . . . 694
5.1.3.2 Factorization Theorem of Polynomials . . . . . . . . . . . . 696
5.1.3.3 Diophantine equation . . . . . . . . . . . . . . . . . . . . . 697
5.1.3.4 First order univariate Polynomial and Equations . . . . . . . 698
5.1.3.5 Second order univariate Polynomial and Equations . . . . . . 699
5.1.3.5.1 Irrational Equations . . . . . . . . . . . . . . . . . 703
5.1.3.5.2 Gold Number . . . . . . . . . . . . . . . . . . . . . 705
5.1.3.6 Third order univariate Polynomial and Equations . . . . . . . 705
5.1.3.7 Fourth order univariate Polynomial and Equations . . . . . . 708
5.1.3.8 Trigonometric Polynomials . . . . . . . . . . . . . . . . . . 711
5.1.3.9 Cyclotomic Polynomials . . . . . . . . . . . . . . . . . . . . 711
5.1.3.10 Legendre Polynomials . . . . . . . . . . . . . . . . . . . . . 714
5.2 Set Algebra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 719
5.2.1 Groups Algebra and Geometry . . . . . . . . . . . . . . . . . . . . . . 719
5.2.1.1 Cyclic Groups . . . . . . . . . . . . . . . . . . . . . . . . . 720
5.2.1.2 Transformations Groups . . . . . . . . . . . . . . . . . . . . 723
5.2.1.3 Group of Symetries . . . . . . . . . . . . . . . . . . . . . . 731
5.2.1.3.1 Orbits and Stabilizers . . . . . . . . . . . . . . . . 735
5.2.1.4 Permutations Groups . . . . . . . . . . . . . . . . . . . . . . 736
5.2.2 Galois Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 746
5.2.2.1 Elementary symmetric and Invariant Polynomials . . . . . . . 746
5.2.2.2 General Vieta’s formulas . . . . . . . . . . . . . . . . . . . . 750
5.3 Differential and Integral Calculus . . . . . . . . . . . . . . . . . . . . . . . . . 751
5.3.1 Differential Calculus . . . . . . . . . . . . . . . . . . . . . . . . . . . 751
5.3.1.1 Differentials . . . . . . . . . . . . . . . . . . . . . . . . . . 760
5.3.1.2 Usual Derivatives . . . . . . . . . . . . . . . . . . . . . . . . 767
5.3.1.3 Smoothness . . . . . . . . . . . . . . . . . . . . . . . . . . . 778
5.3.2 Integral Calculus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 779
5.3.2.1 Definite Integral . . . . . . . . . . . . . . . . . . . . . . . . 779
5.3.2.2 Indefinite Integral . . . . . . . . . . . . . . . . . . . . . . . 785
5.3.2.3 Double Integral . . . . . . . . . . . . . . . . . . . . . . . . . 792
5.3.2.3.1 Fubini’s theorem . . . . . . . . . . . . . . . . . . . 794
5.3.2.4 Integration by Substitution . . . . . . . . . . . . . . . . . . . 795
xii

Draft
5.3.2.4.1 Jacobian . . . . . . . . . . . . . . . . . . . . . . . 796
5.3.2.5 Integration by Parts . . . . . . . . . . . . . . . . . . . . . . . 802
5.3.2.6 Usual Primitives . . . . . . . . . . . . . . . . . . . . . . . . 804
5.3.2.7 Dirac Function . . . . . . . . . . . . . . . . . . . . . . . . . 823
5.3.2.8 Gamma Euler Function . . . . . . . . . . . . . . . . . . . . 825
5.3.2.8.1 Euler-Mascheroni Constant . . . . . . . . . . . . . 829
5.3.2.9 Curvilinear Integrals . . . . . . . . . . . . . . . . . . . . . . 830
5.3.2.9.1 Curvilinear Integral of a scalar field . . . . . . . . . 831
5.3.2.9.2 Curvilinear Integral of a vector field . . . . . . . . . 832
5.3.3 Differential Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . 835
5.3.3.1 First order Differential Equations . . . . . . . . . . . . . . . 836
5.3.3.2 Linear Differential Equations . . . . . . . . . . . . . . . . . 836
5.3.3.3 Resolution Methods of Differential Equations . . . . . . . . . 838
5.3.3.3.1 Method of characteristic polynomial . . . . . . . . 839
5.3.3.3.1.1 Resolution of the H.E. of the first order
L.D.E. with constant coefficients . . . . . . 839
5.3.3.3.1.2 Resolution of the H.E. of the first order
L.D.E. with non-constant coefficients . . . . 840
5.3.3.3.1.3 Resolution of the H.E. of the second order
L.D.E. with constant coefficients . . . . . . 841
5.3.3.3.2 Integrating Factor Method (Euler’s Method) . . . . 845
5.3.3.3.3 Method of separation of variables . . . . . . . . . . 848
5.3.3.3.4 Method of constant variation . . . . . . . . . . . . . 849
5.3.4 Systems of Differential Equations . . . . . . . . . . . . . . . . . . . . 851
5.3.5 Regular Methods of Perturbations . . . . . . . . . . . . . . . . . . . . 856
5.3.5.1 Perturbation theory for algebraic equations . . . . . . . . . . 856
5.3.5.2 Perturbation theory of differential equations . . . . . . . . . . 858
5.4 Sequences and Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 862
5.4.1 Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 862
5.4.1.1 Arithmetic Sequences . . . . . . . . . . . . . . . . . . . . . 863
5.4.1.2 Harmonic Sequences . . . . . . . . . . . . . . . . . . . . . . 866
5.4.1.3 Geometric Sequences . . . . . . . . . . . . . . . . . . . . . 867
5.4.1.4 Cauchy Sequence . . . . . . . . . . . . . . . . . . . . . . . . 868
5.4.1.5 Fibonacci Sequence . . . . . . . . . . . . . . . . . . . . . . 872
5.4.1.6 Logic Sequences/Psychologist Sequences . . . . . . . . . . . 873
5.4.2 Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 874
5.4.2.1 Gauss Series . . . . . . . . . . . . . . . . . . . . . . . . . . 875
5.4.2.1.1 Bernoulli’s Numbers and Polynomials . . . . . . . . 878
5.4.2.2 Arithmetic Series . . . . . . . . . . . . . . . . . . . . . . . . 883
5.4.2.3 Geometric Series . . . . . . . . . . . . . . . . . . . . . . . . 884
5.4.2.3.1 Zeta function and Euler’s identity . . . . . . . . . . 885
5.4.2.4 Taylor and Maclaurin Series . . . . . . . . . . . . . . . . . . 890
5.4.2.4.1 Usual Maclaurin developments . . . . . . . . . . . 895
5.4.2.4.2 Taylor series of bivariate functions (multivariate
Taylor series) . . . . . . . . . . . . . . . . . . . . . 901
5.4.2.4.3 Quadratic Form . . . . . . . . . . . . . . . . . . . 903
5.4.2.4.4 Lagrange Remainder . . . . . . . . . . . . . . . . . 905
5.4.2.4.5 Taylor Series with Integral Remainder . . . . . . . . 907
xiii

Draft
5.4.2.5 Fourier Series . . . . . . . . . . . . . . . . . . . . . . . . . . 909
5.4.2.5.1 Power of a signal . . . . . . . . . . . . . . . . . . . 928
5.4.2.5.2 Fourier Transform . . . . . . . . . . . . . . . . . . 930
5.4.2.6 Bessel Series . . . . . . . . . . . . . . . . . . . . . . . . . . 939
5.4.2.6.1 Zero order Bessel’s Functions . . . . . . . . . . . . 939
5.4.2.6.2 n order Bessel’s Functions . . . . . . . . . . . . . . 939
5.4.2.6.3 Bessel’s Differential Equations of ordern n . . . . . 944
5.4.3 Convergence Criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . 945
5.4.3.1 Integral Test . . . . . . . . . . . . . . . . . . . . . . . . . . 946
5.4.3.2 D’Alembert Rule . . . . . . . . . . . . . . . . . . . . . . . . 947
5.4.3.3 Alternating Series Test . . . . . . . . . . . . . . . . . . . . . 948
5.4.3.4 Fixed Point Theorem . . . . . . . . . . . . . . . . . . . . . . 949
5.5 Vector Calculus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 952
5.5.1 Concept of Arrow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 953
5.5.2 Set of Vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 954
5.5.2.1 Pseudo-Vectors . . . . . . . . . . . . . . . . . . . . . . . . . 956
5.5.2.2 Multiplication by a scalar . . . . . . . . . . . . . . . . . . . 958
5.5.2.2.1 Rule of three . . . . . . . . . . . . . . . . . . . . . 958
5.5.3 Vector Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 960
5.5.3.1 Linear Combinations . . . . . . . . . . . . . . . . . . . . . . 961
5.5.3.2 Sub-vector spaces . . . . . . . . . . . . . . . . . . . . . . . 961
5.5.3.3 Generating families . . . . . . . . . . . . . . . . . . . . . . 962
5.5.3.4 Linear Dependance or Independance . . . . . . . . . . . . . 962
5.5.3.5 Base of a vectorial space . . . . . . . . . . . . . . . . . . . . 963
5.5.3.6 Direction Angles . . . . . . . . . . . . . . . . . . . . . . . . 965
5.5.3.7 Dimensions of a vector space . . . . . . . . . . . . . . . . . 966
5.5.3.8 Extension of a free family . . . . . . . . . . . . . . . . . . . 966
5.5.3.9 Rank of a ﬁnite family . . . . . . . . . . . . . . . . . . . . . 967
5.5.3.10 Direct Sums . . . . . . . . . . . . . . . . . . . . . . . . . . 968
5.5.3.11 Afﬁne spaces . . . . . . . . . . . . . . . . . . . . . . . . . . 969
5.5.4 Euclidean Vectore Spaces . . . . . . . . . . . . . . . . . . . . . . . . . 972
5.5.4.1 Scalar Product (Dot Product) . . . . . . . . . . . . . . . . . 974
5.5.4.1.1 Cauchy–Schwarz inequality . . . . . . . . . . . . . 978
5.5.4.1.2 Triangular Inequalities . . . . . . . . . . . . . . . . 979
5.5.4.1.3 General Scalar/Dot Product . . . . . . . . . . . . . 980
5.5.4.2 Cross Product . . . . . . . . . . . . . . . . . . . . . . . . . 981
5.5.4.3 Mixed Product (triple product9 . . . . . . . . . . . . . . . . 986
5.5.5 Vectorial Functional Space . . . . . . . . . . . . . . . . . . . . . . . . 988
5.5.6 Hermitian Vector Space . . . . . . . . . . . . . . . . . . . . . . . . . . 990
5.5.6.1 Hermitian Inner Product . . . . . . . . . . . . . . . . . . . . 991
5.5.6.2 Types of Vectors Spaces . . . . . . . . . . . . . . . . . . . . 992
5.5.7 System of Coordinates . . . . . . . . . . . . . . . . . . . . . . . . . . 993
5.5.7.1 Cartesian (rectangular) Coordinate System . . . . . . . . . . 993
5.5.7.2 Spherical Coordinate System . . . . . . . . . . . . . . . . . 994
5.5.7.3 Cylindrical Coordinate System . . . . . . . . . . . . . . . . 999
5.5.7.4 Polar Coordinate System . . . . . . . . . . . . . . . . . . . . 1001
5.5.8 Differential Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . 1002
5.5.8.1 Gradients of Scalar Field . . . . . . . . . . . . . . . . . . . . 1004
xiv

Draft
5.5.8.2 Gradients of Vector Field . . . . . . . . . . . . . . . . . . . 1010
5.5.8.3 Divergences of a Vector Field . . . . . . . . . . . . . . . . . 1011
5.5.8.4 Rotationals of a Vector Field (Curl) . . . . . . . . . . . . . . 1019
5.5.8.5 Laplacians of Scalar Field (Laplace Operator) . . . . . . . . . 1029
5.5.8.6 Laplacians of a Vector Field . . . . . . . . . . . . . . . . . . 1033
5.5.8.7 Remarkable Identities . . . . . . . . . . . . . . . . . . . . . 1041
5.5.8.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1044
5.6 Linear Algebra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1048
5.6.0.1 Linear Systems . . . . . . . . . . . . . . . . . . . . . . . . . 1052
5.6.1 Linear Transformations . . . . . . . . . . . . . . . . . . . . . . . . . . 1056
5.6.2 Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1058
5.6.2.1 Rank of a matrix . . . . . . . . . . . . . . . . . . . . . . . . 1059
5.6.2.2 Matrix Algebra . . . . . . . . . . . . . . . . . . . . . . . . . 1064
5.6.2.3 Type of Matrices . . . . . . . . . . . . . . . . . . . . . . . . 1067
5.6.2.4 Determinant . . . . . . . . . . . . . . . . . . . . . . . . . . 1075
5.6.2.4.1 Derivative of a Determinant . . . . . . . . . . . . . 1087
5.6.2.4.2 Determinant Cofactor and Matrix Inverse . . . . . . 1088
5.6.3 Change of Basis (frames) . . . . . . . . . . . . . . . . . . . . . . . . . 1090
5.6.4 Eigenvalues and Eigenvectors . . . . . . . . . . . . . . . . . . . . . . 1093
5.6.4.1 Rotation Matrices and Eigenvalues . . . . . . . . . . . . . . 1096
5.6.5 Spectral Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1098
5.7 Tensor Calculus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1104
5.7.1 Tensor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1105
5.7.2 Indicial Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1107
5.7.2.1 Summation on multiple index . . . . . . . . . . . . . . . . . 1108
5.7.2.2 Kronecker Symbol . . . . . . . . . . . . . . . . . . . . . . . 1108
5.7.2.3 Antisymmetric Symbol (Levi-Civita symbol) . . . . . . . . . 1109
5.7.3 Metric and Signature . . . . . . . . . . . . . . . . . . . . . . . . . . . 1115
5.7.4 Gram’s Determinant . . . . . . . . . . . . . . . . . . . . . . . . . . . 1118
5.7.5 Contravariant and Covariant Components . . . . . . . . . . . . . . . . 1123
5.7.6 Operation in Basis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1124
5.7.6.1 Gram-Schmidt Orthogonalization Method . . . . . . . . . . . 1125
5.8 Spinor Calculus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1126
5.8.1 Unit Spinor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1126
5.8.2 Geometric Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . 1133
5.8.2.1 Plane Symmetries . . . . . . . . . . . . . . . . . . . . . . . 1133
5.8.2.2 Rotations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1136
5.8.2.3 Properties of Pauli Matrices . . . . . . . . . . . . . . . . . . 1141
6 Analysis 1148
6.1 Functional Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1149
6.1.1 Representations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1150
6.1.1.1 Tabular Representation . . . . . . . . . . . . . . . . . . . . . 1150
6.1.1.2 Graphical Representation . . . . . . . . . . . . . . . . . . . 1151
6.1.1.2.1 2D representations . . . . . . . . . . . . . . . . . . 1152
6.1.1.2.2 3D representations . . . . . . . . . . . . . . . . . . 1156
6.1.1.2.3 2D Vector representations . . . . . . . . . . . . . . 1162
6.1.1.2.4 Properties of visual representations . . . . . . . . . 1165
xv

Draft
6.1.1.3 Analytical Representation . . . . . . . . . . . . . . . . . . . 1170
6.1.2 Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1173
6.1.2.1 Limits and Continuity of Functions . . . . . . . . . . . . . . 1184
6.1.2.2 Asymptotes . . . . . . . . . . . . . . . . . . . . . . . . . . . 1190
6.1.3 Logarithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1196
6.1.4 Transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1203
6.1.4.1 Fourier Transform . . . . . . . . . . . . . . . . . . . . . . . 1203
6.1.4.2 Laplace Transform . . . . . . . . . . . . . . . . . . . . . . . 1203
6.1.4.3 Hilbert Transform . . . . . . . . . . . . . . . . . . . . . . . 1204
6.1.5 Functional dot product (inner product) . . . . . . . . . . . . . . . . . . 1204
6.2 Complex Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1209
6.2.1 Linear Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1209
6.2.2 Holomorphic Functions . . . . . . . . . . . . . . . . . . . . . . . . . . 1219
6.2.2.1 Orthogonality of real and imaginary iso-curves . . . . . . . . 1225
6.2.3 Complex Logarithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1227
6.2.4 Complex Integral Calculus . . . . . . . . . . . . . . . . . . . . . . . . 1230
6.2.4.1 Convergence of a complex series . . . . . . . . . . . . . . . 1237
6.2.5 Path Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1246
6.2.5.1 Inverse Path . . . . . . . . . . . . . . . . . . . . . . . . . . 1247
6.2.6 Laurent Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1249
6.2.7 Singularities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1257
6.2.8 Residue Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1260
6.2.8.1 Pole at inﬁnity . . . . . . . . . . . . . . . . . . . . . . . . . 1264
6.3 Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1266
6.3.1 General Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1267
6.3.1.1 Topological Spaces . . . . . . . . . . . . . . . . . . . . . . . 1268
6.3.2 Metric Space and Distance . . . . . . . . . . . . . . . . . . . . . . . . 1269
6.3.2.1 Equivalent Distances . . . . . . . . . . . . . . . . . . . . . . 1274
6.3.2.2 Lipschitz Functions . . . . . . . . . . . . . . . . . . . . . . 1274
6.3.2.3 Continuity and Uniform Continuity . . . . . . . . . . . . . . 1277
6.3.3 Opened and Closed Set . . . . . . . . . . . . . . . . . . . . . . . . . . 1279
6.3.3.1 Balls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1280
6.3.3.2 Partititions . . . . . . . . . . . . . . . . . . . . . . . . . . . 1284
6.3.3.3 Formal Ball . . . . . . . . . . . . . . . . . . . . . . . . . . . 1286
6.3.3.4 Diameter . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1287
6.3.4 Varieties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1289
6.3.4.1 Surfaces Homeomorphism . . . . . . . . . . . . . . . . . . . 1290
6.3.4.2 Differential Varieties . . . . . . . . . . . . . . . . . . . . . . 1294
6.4 Measure Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1296
6.4.1 Measurable Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1296
6.4.1.1 Monotone Classes . . . . . . . . . . . . . . . . . . . . . . . 1304
7 Geometry 1307
7.1 Trigonometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1309
7.1.1 Radian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1309
7.1.2 Circle Trigonometry . . . . . . . . . . . . . . . . . . . . . . . . . . . 1311
7.1.2.1 Remarkable trigonometric triangle identities . . . . . . . . . 1319
7.1.2.1.1 Laws of Cosines . . . . . . . . . . . . . . . . . . . 1323
xvi

Draft
7.1.2.1.2 Laws of Sines . . . . . . . . . . . . . . . . . . . . 1325
7.1.3 Hyperbolic Trigonometry . . . . . . . . . . . . . . . . . . . . . . . . . 1326
7.1.3.1 Remarkable hyperbolic identities . . . . . . . . . . . . . . . 1333
7.1.4 Spherical Trigonometry . . . . . . . . . . . . . . . . . . . . . . . . . . 1335
7.1.5 Solide Angle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1340
7.2 Euclidean Geometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1343
7.2.1 Objects of Euclidean Geometry . . . . . . . . . . . . . . . . . . . . . . 1343
7.2.1.1 Dimensions . . . . . . . . . . . . . . . . . . . . . . . . . . . 1345
7.2.2 Euclid Constructions . . . . . . . . . . . . . . . . . . . . . . . . . . . 1351
7.2.2.1 Segments and Lines . . . . . . . . . . . . . . . . . . . . . . 1353
7.2.2.1.1 Quantities of the same type . . . . . . . . . . . . . 1353
7.2.3 Plane Geometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1358
7.2.3.1 Displacements and Turnarounds . . . . . . . . . . . . . . . . 1359
7.2.3.2 Plane angles . . . . . . . . . . . . . . . . . . . . . . . . . . 1360
7.2.3.2.1 Angle Measurements . . . . . . . . . . . . . . . . . 1366
7.2.3.2.2 Units of Angle Measurements . . . . . . . . . . . . 1369
7.2.3.2.3 Bisector . . . . . . . . . . . . . . . . . . . . . . . 1372
7.2.3.3 Triangles . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1373
7.2.3.3.1 Equal Triangles (congruent triangles) . . . . . . . . 1374
7.2.3.3.2 Isosceles Triangles . . . . . . . . . . . . . . . . . . 1377
7.3 Non-Euclidean Geometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1382
7.3.1 Geodesic and Metric Equation . . . . . . . . . . . . . . . . . . . . . . 1383
7.3.2 Riemann Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1387
7.4 Projective Geometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1393
7.4.1 Conical Perspective (Central Perspective) . . . . . . . . . . . . . . . . 1394
7.4.1.1 Images of Points . . . . . . . . . . . . . . . . . . . . . . . . 1397
7.4.1.2 Images of Straight Lines . . . . . . . . . . . . . . . . . . . . 1413
7.5 Analytical Geometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1415
7.5.1 Conics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1415
7.5.1.1 Algebraic approach . . . . . . . . . . . . . . . . . . . . . . . 1416
7.5.1.2 Geometric Approach . . . . . . . . . . . . . . . . . . . . . . 1424
7.5.1.3 Dudelin Theorem (Dudelin Spheres) . . . . . . . . . . . . . 1432
7.5.2 Parametrizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1434
7.5.2.1 Equation of the Plane . . . . . . . . . . . . . . . . . . . . . 1434
7.5.2.2 Equation of the straight line . . . . . . . . . . . . . . . . . . 1437
7.5.2.2.1 Distance from a line to a point . . . . . . . . . . . . 1439
7.5.2.2.2 Line deﬁned by the intersection of planes . . . . . . 1440
7.5.2.2.3 Parametric equation of a line in R3
. . . . . . . . . 1440
7.5.2.3 Equation of a square . . . . . . . . . . . . . . . . . . . . . . 1442
7.5.2.4 Equation of a cone . . . . . . . . . . . . . . . . . . . . . . . 1443
7.5.2.5 Equation of a Sphere . . . . . . . . . . . . . . . . . . . . . . 1445
7.5.2.6 Equation of an Ellipsoid (spheroid, geoid) . . . . . . . . . . . 1447
7.5.2.7 Equation of a Cylinder . . . . . . . . . . . . . . . . . . . . . 1449
7.5.2.8 Surface of revolution . . . . . . . . . . . . . . . . . . . . . . 1450
7.5.2.8.1 Paraboloid . . . . . . . . . . . . . . . . . . . . . . 1451
7.5.2.8.2 Hyperboloid . . . . . . . . . . . . . . . . . . . . . 1453
7.5.2.8.3 Torus . . . . . . . . . . . . . . . . . . . . . . . . . 1455
7.6 Differential Geometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1457
xvii

Draft
7.6.1 Parametric Curves . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1457
7.6.2 Isolines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1463
7.6.3 Frenet Frame . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1469
7.6.4 Surface Patchs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1480
7.6.4.1 Metric of a Surface Patch . . . . . . . . . . . . . . . . . . . 1481
7.6.4.1.1 Regularity of a Surface . . . . . . . . . . . . . . . . 1483
7.7 Geometric Shapes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1487
7.7.1 Known Surfaces (Areas) . . . . . . . . . . . . . . . . . . . . . . . . . 1488
7.7.1.1 Polygons . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1488
7.7.1.2 Rectangle . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1493
7.7.1.3 Square . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1494
7.7.1.4 Unspeciﬁed Triangle . . . . . . . . . . . . . . . . . . . . . . 1495
7.7.1.5 Isosceles Triangle . . . . . . . . . . . . . . . . . . . . . . . 1499
7.7.1.6 Equilateral Triangle . . . . . . . . . . . . . . . . . . . . . . 1501
7.7.1.7 Right Triangle . . . . . . . . . . . . . . . . . . . . . . . . . 1503
7.7.1.8 Trapezoid . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1504
7.7.1.9 Parallelogram . . . . . . . . . . . . . . . . . . . . . . . . . . 1506
7.7.1.10 Hexagon . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1508
7.7.1.11 Rhombus . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1512
7.7.1.12 Circle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1514
7.7.1.13 Ellipse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1516
7.7.2 Known Volumes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1519
7.7.2.1 Polyhedron . . . . . . . . . . . . . . . . . . . . . . . . . . . 1520
7.7.2.1.1 Parallelepiped . . . . . . . . . . . . . . . . . . . . 1520
7.7.2.1.1.1 Moment of Inertia of a rectangular plate . . . 1522
7.7.2.1.1.2 Moment of Inertia of a triangular plate . . . 1524
7.7.2.1.2 Pyramid . . . . . . . . . . . . . . . . . . . . . . . 1526
7.7.2.1.2.1 Moment of Inertia of a regular square pyramid1527
7.7.2.1.3 Right Prism . . . . . . . . . . . . . . . . . . . . . . 1528
7.7.2.1.4 Regular Polyhedron . . . . . . . . . . . . . . . . . 1529
7.7.2.1.5 Regular Tetrahedron . . . . . . . . . . . . . . . . . 1534
7.7.2.1.6 Regular hexahedron (cube) . . . . . . . . . . . . . 1537
7.7.2.1.7 Regular octahedron . . . . . . . . . . . . . . . . . 1537
7.7.2.1.8 Regular Icosahedron . . . . . . . . . . . . . . . . . 1541
7.7.2.1.9 Regular Dodecahedron . . . . . . . . . . . . . . . . 1545
7.7.2.2 Solids of Revolution . . . . . . . . . . . . . . . . . . . . . . 1550
7.7.2.2.1 Cylinder . . . . . . . . . . . . . . . . . . . . . . . 1552
7.7.2.2.2 Cone . . . . . . . . . . . . . . . . . . . . . . . . . 1554
7.7.2.2.3 Sphere . . . . . . . . . . . . . . . . . . . . . . . . 1556
7.7.2.2.4 Torus . . . . . . . . . . . . . . . . . . . . . . . . . 1560
7.8 Graph Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1562
7.8.1 Type of Graphs and Structures . . . . . . . . . . . . . . . . . . . . . . 1563
7.8.2 Graph Adjacency matrix . . . . . . . . . . . . . . . . . . . . . . . . . 1585
7.8.3 Categories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1590
7.9 Knot Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1594
7.9.1 Braids Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . 1594
7.9.1.1 Braids Group . . . . . . . . . . . . . . . . . . . . . . . . . . 1596
7.9.2 Knot Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . 1599
xviii

Draft
7.9.2.1 Knots Group . . . . . . . . . . . . . . . . . . . . . . . . . . 1601
7.9.3 Tait’s Knot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1604
7.9.4 Mathematical Formalisation . . . . . . . . . . . . . . . . . . . . . . . 1608
7.9.4.1 Planar Representation . . . . . . . . . . . . . . . . . . . . . 1616
8 Mechanics 1618
8.1 Principia (Analytical/Lagrangian Mechanics) . . . . . . . . . . . . . . . . . . 1620
8.1.1 System of Units . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1623
8.1.1.1 Dimensional Analysis . . . . . . . . . . . . . . . . . . . . . 1630
8.1.1.1.1 Time . . . . . . . . . . . . . . . . . . . . . . . . . 1632
8.1.1.1.2 Length . . . . . . . . . . . . . . . . . . . . . . . . 1633
8.1.1.1.3 Mass . . . . . . . . . . . . . . . . . . . . . . . . . 1635
8.1.1.1.4 Energy . . . . . . . . . . . . . . . . . . . . . . . . 1638
8.1.1.1.5 Electric Charge . . . . . . . . . . . . . . . . . . . . 1639
8.1.1.2 Scientiﬁc Notation and Metric Preﬁxes . . . . . . . . . . . . 1642
8.1.2 Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1645
8.2 Analytical/Lagrangian Mechanics . . . . . . . . . . . . . . . . . . . . . . . . 1647
8.2.1 Lagrangian formalism . . . . . . . . . . . . . . . . . . . . . . . . . . 1649
8.2.1.1 Generalized coordinates and frames . . . . . . . . . . . . . . 1649
8.2.1.2 Variational Principle . . . . . . . . . . . . . . . . . . . . . . 1653
8.2.1.3 Euler-Lagrange Equation . . . . . . . . . . . . . . . . . . . . 1655
8.2.1.3.1 Beltrami Identity . . . . . . . . . . . . . . . . . . . 1661
8.2.1.3.2 Theorem of Variational Calculus . . . . . . . . . . 1662
8.2.2 Canonical Formalism . . . . . . . . . . . . . . . . . . . . . . . . . . . 1664
8.3 Classical Mechanics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1665
8.3.1 Newton’s Laws . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1669
8.3.1.1 Newton First Law (Inertia Law) . . . . . . . . . . . . . . . . 1669
8.3.1.2 Newton Second Law (Fundamental Principle of Dynamics) . 1671
8.3.1.3 Newton Third Law (Law of Action and Reatction) . . . . . . 1673
8.3.2 Center of Mass and Reduced Weight . . . . . . . . . . . . . . . . . . . 1674
8.3.2.1 Center of Mass Theorem . . . . . . . . . . . . . . . . . . . . 1677
8.3.2.2 Guldin’s Theorem . . . . . . . . . . . . . . . . . . . . . . . 1679
8.3.3 Kinematics of Rectilinear Motion . . . . . . . . . . . . . . . . . . . . 1682
8.3.3.1 Position . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1682
8.3.3.2 Velocity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1683
8.3.3.3 Acceleration . . . . . . . . . . . . . . . . . . . . . . . . . . 1684
8.3.3.3.1 Osculator Plane . . . . . . . . . . . . . . . . . . . 1687
8.3.3.4 Galilean Relativity Principle . . . . . . . . . . . . . . . . . . 1689
8.4 Wave Mechanics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1692
8.4.0.1 Wave Function . . . . . . . . . . . . . . . . . . . . . . . . . 1692
8.4.0.2 Wave Equation . . . . . . . . . . . . . . . . . . . . . . . . . 1693
8.4.1 Type of Waves . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1695
8.4.1.1 Periodic Waves . . . . . . . . . . . . . . . . . . . . . . . . . 1695
8.4.1.2 Harmonic Waves . . . . . . . . . . . . . . . . . . . . . . . . 1696
8.4.1.3 Stationnary Waves . . . . . . . . . . . . . . . . . . . . . . . 1697
8.4.1.4 Vibration Modes in a Stretch String . . . . . . . . . . . . . . 1699
8.4.1.4.1 Dirichlet Conditions . . . . . . . . . . . . . . . . . 1701
8.4.1.4.2 Neumann Conditions . . . . . . . . . . . . . . . . . 1707
xix

Draft
8.5 Statistical Mechanics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1709
8.5.1 Statistical Information Theory . . . . . . . . . . . . . . . . . . . . . . 1709
8.5.2 Boltzmann Law . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1716
8.5.3 Statistical Physics Distributions . . . . . . . . . . . . . . . . . . . . . 1721
8.5.3.1 Maxwell Distribution (velocity distribution) . . . . . . . . . . 1721
8.5.3.2 Maxwell-Boltzmann Distribution . . . . . . . . . . . . . . . 1727
8.6 Thermodynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1731
8.6.1 Thermodynamic Variables . . . . . . . . . . . . . . . . . . . . . . . . 1732
8.6.2 Thermodynamics Systems . . . . . . . . . . . . . . . . . . . . . . . . 1735
8.6.3 Thermodynamic Transformations . . . . . . . . . . . . . . . . . . . . 1736
8.6.4 State Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1738
8.6.4.1 Phases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1740
8.6.5 Equation of State . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1742
8.6.5.1 Ideal Gaz Law . . . . . . . . . . . . . . . . . . . . . . . . . 1742
8.6.5.2 State equation of a Liquid . . . . . . . . . . . . . . . . . . . 1743
8.6.5.3 State equation of a Solide . . . . . . . . . . . . . . . . . . . 1745
8.7 Continuum Mechanics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1748
8.7.1 Rigid Bodies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1749
8.7.1.1 Pressures . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1749
8.7.1.2 Elasticity of Solids . . . . . . . . . . . . . . . . . . . . . . . 1750
8.7.1.2.1 Hooke’s law . . . . . . . . . . . . . . . . . . . . . 1755
8.7.1.2.2 Shear Modulus . . . . . . . . . . . . . . . . . . . . 1759
8.7.2 Liquids . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1760
8.7.3 Gas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1761
9 Electromagnetism 1763
9.1 Electrostatics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1765
9.1.1 Electric Force . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1765
9.1.2 Electric Potential . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1769
9.1.2.1 Path Independance . . . . . . . . . . . . . . . . . . . . . . . 1772
9.1.3 Equipotential and Field lines . . . . . . . . . . . . . . . . . . . . . . . 1772
9.1.3.1 Infinite straight wire . . . . . . . . . . . . . . . . . . . . . . 1775
9.1.3.2 Electric Rigid Dipole . . . . . . . . . . . . . . . . . . . . . . 1777
9.1.4 Electric Field Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1788
9.1.4.1 Capacitor . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1789
9.2 Magnetostatics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1792
9.2.1 Ampere’s theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1795
9.2.1.1 Infinitely long solenoid . . . . . . . . . . . . . . . . . . . . 1797
9.2.1.2 Toroidal coils . . . . . . . . . . . . . . . . . . . . . . . . . . 1798
9.2.1.3 Electromagnet . . . . . . . . . . . . . . . . . . . . . . . . . 1799
9.2.1.3.1 Strength of a magnet or electromagnet . . . . . . . 1800
9.2.2 Maxwell-Ampere Relation . . . . . . . . . . . . . . . . . . . . . . . . 1802
9.2.3 Biot-Savart law . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1803
9.2.3.1 Magnetic field for a current loop . . . . . . . . . . . . . . . . 1804
9.2.3.2 Magnetic field for an infinite wire . . . . . . . . . . . . . . . 1806
9.2.4 Magnetic dipole . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1808
9.3 Electrodynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1814
9.3.1 Maxwell Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1815
xx

Draft
9.3.1.1 First Maxwell Equation (constant electric flow) . . . . . . . . 1815
9.3.1.2 Second Maxwell Equation (non-existence of magnetic
monopomple) . . . . . . . . . . . . . . . . . . . . . . . . . . 1818
9.3.1.3 Third Maxwell Equation . . . . . . . . . . . . . . . . . . . . 1819
9.3.1.3.1 Betatron . . . . . . . . . . . . . . . . . . . . . . . 1822
9.3.1.4 Fourth Maxwell Equation . . . . . . . . . . . . . . . . . . . 1826
9.3.1.5 Magnetic Monopoles . . . . . . . . . . . . . . . . . . . . . . 1829
9.3.2 Charge conservation equation . . . . . . . . . . . . . . . . . . . . . . 1831
9.3.3 Gauge Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1832
9.4 Electrokinetics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1837
9.4.1 Kirchoff’s laws . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1838
9.4.1.1 Mesh law . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1838
9.4.1.2 Nodes law . . . . . . . . . . . . . . . . . . . . . . . . . . . 1839
9.4.2 Drude model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1839
9.4.3 Ohm’s law . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1845
9.4.3.1 Equivalent Resistance . . . . . . . . . . . . . . . . . . . . . 1847
9.4.3.2 Equivalent Capacities . . . . . . . . . . . . . . . . . . . . . 1849
9.4.4 Electromotive Force . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1850
9.5 Geometric Optics (ray optics) . . . . . . . . . . . . . . . . . . . . . . . . . . . 1853
9.5.1 Sources and Shadows . . . . . . . . . . . . . . . . . . . . . . . . . . . 1853
9.5.2 Colors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1857
9.5.3 Radiometry/Photometry . . . . . . . . . . . . . . . . . . . . . . . . . 1863
9.5.3.1 Energy flow . . . . . . . . . . . . . . . . . . . . . . . . . . . 1863
9.5.3.1.1 Beer–Lambert law . . . . . . . . . . . . . . . . . . 1863
9.5.3.2 Light Intensity (Radiant Intensity) . . . . . . . . . . . . . . . 1865
9.5.3.3 Energy Emittance (Radiant Emittance) . . . . . . . . . . . . 1866
9.5.3.4 Radiance and Luminance . . . . . . . . . . . . . . . . . . . 1867
9.5.3.4.1 Lambert’s Law . . . . . . . . . . . . . . . . . . . . 1868
9.5.3.5 Kirchhoff’s law of Radiation . . . . . . . . . . . . . . . . . . 1870
9.5.3.6 Spectral Decomposition . . . . . . . . . . . . . . . . . . . . 1870
9.5.4 Law of Refraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1871
9.6 Wave Optics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1874
9.6.1 Huygens’ principle . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1874
9.6.2 Fraunhofer Diffraction . . . . . . . . . . . . . . . . . . . . . . . . . . 1877
9.6.2.1 Case of a rectangular slot . . . . . . . . . . . . . . . . . . . 1877
10 Atomistic 1884
10.1 Corpuscular Quantum Physics . . . . . . . . . . . . . . . . . . . . . . . . . . 1885
10.1.1 Dalton’s model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1886
10.1.2 Thomson’s model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1887
10.1.3 Rhuterfords’s model . . . . . . . . . . . . . . . . . . . . . . . . . . . 1888
10.1.4 Bohr’s Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1890
10.1.4.1 Bohr’s Postulates . . . . . . . . . . . . . . . . . . . . . . . . 1890
10.1.4.2 Quantification . . . . . . . . . . . . . . . . . . . . . . . . . 1890
10.1.4.3 Hydrogen Type Atoms Model without dragging . . . . . . . . 1893
10.1.4.4 Hydrogen Type Atoms Model with dragging . . . . . . . . . 1897
10.1.4.5 Neutron Assumption . . . . . . . . . . . . . . . . . . . . . . 1900
10.1.5 Wilson and Sommerfeld’s Model . . . . . . . . . . . . . . . . . . . . . 1902
xxi

Draft
10.1.6 Relativistic Sommerfeld Model . . . . . . . . . . . . . . . . . . . . . . 1906
10.2 Wave Quantum Physics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1914
10.2.1 Postulates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1915
10.2.1.1 1st Postulate: Quantum State . . . . . . . . . . . . . . . . . 1916
10.2.1.2 2nd postulate: Time evolution of a quantum state . . . . . . . 1918
10.2.1.3 3rd postulate: Observables and operators . . . . . . . . . . . 1919
10.2.1.4 4th postulate: Measure of a property) . . . . . . . . . . . . . 1921
10.2.1.5 5th postulate: Average of a property) . . . . . . . . . . . . . 1923
10.2.2 Classical principles of uncertainty . . . . . . . . . . . . . . . . . . . . 1924
10.3 Relativistic Quantum Physics . . . . . . . . . . . . . . . . . . . . . . . . . . . 1925
10.3.1 Relativistic Schrödinger evolution equation . . . . . . . . . . . . . . . 1926
10.3.1.1 Antimatter . . . . . . . . . . . . . . . . . . . . . . . . . . . 1927
10.3.2 Generalized Klein-Gordon Equation . . . . . . . . . . . . . . . . . . . 1930
10.4 Nuclear Physics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1934
10.4.1 Nuclear Weapon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1934
10.4.2 Radioactivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1937
10.4.2.1 Disintegration . . . . . . . . . . . . . . . . . . . . . . . . . 1940
10.4.2.1.1 Half-life isotope . . . . . . . . . . . . . . . . . . . 1941
10.4.2.2 Activity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1941
10.5 Quantum Field Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1944
10.5.1 Yukawa potential . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1947
10.5.1.1 Mass ﬁelds . . . . . . . . . . . . . . . . . . . . . . . . . . . 1949
10.6 Elementary Particle Physics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1951
10.6.1 Coupling Constants . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1952
11 Cosmology 1957
11.1 Astronomy (Celestial Mechanics) . . . . . . . . . . . . . . . . . . . . . . . . . 1958
11.1.1 Drake Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1958
11.1.2 Kepler’s Laws . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1959
11.1.2.1 First Kepler’s Law (conicity law) . . . . . . . . . . . . . . . 1959
11.1.2.2 Second Kepler’s Law (area law) . . . . . . . . . . . . . . . . 1960
11.1.2.3 Third Kepler’s Law (periods’ law) . . . . . . . . . . . . . . . 1962
11.1.3 Newton Gravitational Law . . . . . . . . . . . . . . . . . . . . . . . . 1965
11.1.3.1 Shell Theorem . . . . . . . . . . . . . . . . . . . . . . . . . 1971
11.1.3.2 Asteroids/Meteors impact velocity . . . . . . . . . . . . . . . 1972
11.1.3.3 Spherisation of Celestial Bodies . . . . . . . . . . . . . . . . 1973
11.1.3.3.1 Flattening of Celestial Bodies (rotational ﬂattening) 1974
11.1.3.4 Stability of Atmospheres . . . . . . . . . . . . . . . . . . . . 1976
11.1.4 Roche’s Limit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1977
11.1.5 Keplerian Orbitals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1979
11.1.5.1 First Binet Formula . . . . . . . . . . . . . . . . . . . . . . . 1979
11.2 Astrophysics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1981
11.2.1 Stars . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1981
11.2.1.1 Star Formation and Genesis . . . . . . . . . . . . . . . . . . 1987
11.2.1.1.1 Collapse of an Interstellar Cloud . . . . . . . . . . . 1987
11.2.1.1.2 Nuclear Duration Life . . . . . . . . . . . . . . . . 1989
11.3 Special Relativity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1994
11.3.1 Assumptions and Principles . . . . . . . . . . . . . . . . . . . . . . . 1995
xxii

Draft
11.3.1.1 Postulate of Invariance . . . . . . . . . . . . . . . . . . . . . 1995
11.3.1.2 Cosmological principle . . . . . . . . . . . . . . . . . . . . . 1995
11.3.1.3 Special Relativity Principle . . . . . . . . . . . . . . . . . . 1996
11.3.2 Lorentz Transformations/Boost . . . . . . . . . . . . . . . . . . . . . . 1998
11.3.2.1 Displacement four-vector . . . . . . . . . . . . . . . . . . . 2002
11.3.2.1.1 Wave Equation Invariance . . . . . . . . . . . . . . 2004
11.3.2.1.2 Wave Equation Invariance . . . . . . . . . . . . . . 2005
11.4 General Relativity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2008
11.4.1 Assumptions and Principles . . . . . . . . . . . . . . . . . . . . . . . 2008
11.4.1.1 Equivalence Postulates . . . . . . . . . . . . . . . . . . . . . 2008
11.4.1.2 Mach Principle . . . . . . . . . . . . . . . . . . . . . . . . . 2012
11.4.2 Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2013
11.4.2.1 Schild Critera . . . . . . . . . . . . . . . . . . . . . . . . . . 2018
11.5 Cosmology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2021
11.5.0.1 Newtonian Cosmological Model . . . . . . . . . . . . . . . . 2021
11.5.0.2 Hubble’s Law . . . . . . . . . . . . . . . . . . . . . . . . . . 2022
11.5.1 Friedmann Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . 2025
11.5.1.0.1 Critical Density . . . . . . . . . . . . . . . . . . . 2029
11.6 String Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2032
11.6.1 Wave equation of a transervsal string . . . . . . . . . . . . . . . . . . . 2033
11.6.2 Non-relativistic Wave equation of a transversal string . . . . . . . . . . 2035
11.6.2.1 Nambu-Goto Action . . . . . . . . . . . . . . . . . . . . . . 2041
11.6.3 Lagrangian of a String . . . . . . . . . . . . . . . . . . . . . . . . . . 2048
12 Chemistry 2049
12.1 Quantum Chemistry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2050
12.1.1 Inﬁnite three-dimensional rectangular potential . . . . . . . . . . . . . 2050
12.1.2 Molecular Vibrations . . . . . . . . . . . . . . . . . . . . . . . . . . . 2053
12.1.3 Hydrogenoid Atom . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2056
12.1.4 Rigid Rotator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2060
12.2 Molecular Chemistry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2076
12.2.0.1 Orbital Approximations . . . . . . . . . . . . . . . . . . . . 2077
12.2.1 LCAO Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2081
12.3 Analytical Chemistry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2090
12.3.1 Simple Mixtures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2091
12.3.2 Reactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2093
12.4 Thermochemistry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2098
12.4.0.1 Chemical transformations . . . . . . . . . . . . . . . . . . . 2098
12.4.1 Molar Quantities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2100
13 Theoretical Computing 2106
13.1 Numerical Methods/Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 2108
13.1.1 Computer Representation of Numbers . . . . . . . . . . . . . . . . . . 2110
13.1.1.1 Decimal System . . . . . . . . . . . . . . . . . . . . . . . . 2110
13.1.1.2 Binary system . . . . . . . . . . . . . . . . . . . . . . . . . 2111
13.1.1.2.1 Binary arithemetics . . . . . . . . . . . . . . . . . 2112
13.1.1.3 Hexadecimal System . . . . . . . . . . . . . . . . . . . . . . 2112
13.1.1.4 Octal System . . . . . . . . . . . . . . . . . . . . . . . . . . 2113
13.1.1.5 Conversion of decimal system to non-decimal system: . . . . 2113
xxiii

Draft
13.1.2 Complexity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2115
13.1.2.1 NP-Completude . . . . . . . . . . . . . . . . . . . . . . . . 2120
13.1.3 Integer Part . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2122
13.1.4 Heron’s Square Root Algorithm . . . . . . . . . . . . . . . . . . . . . 2124
13.1.5 Archimedes Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . 2126
13.1.6 Euler’s Number e . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2128
13.1.7 Stirling’s factorial approximation . . . . . . . . . . . . . . . . . . . . . 2128
13.1.8 Linear System of Equations . . . . . . . . . . . . . . . . . . . . . . . . 2130
13.1.8.1 One equation with on unknown . . . . . . . . . . . . . . . . 2130
13.1.8.2 Two equations with two unknowns . . . . . . . . . . . . . . 2130
13.1.8.3 Three equations with three unknowns . . . . . . . . . . . . . 2132
13.1.8.4 n equations with n unknowns . . . . . . . . . . . . . . . . . 2135
13.1.9 Polynomials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2136
13.1.10Regression Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . 2139
13.1.10.1 Univariate linear regression model . . . . . . . . . . . . . . . 2141
13.1.10.1.1 Regression line . . . . . . . . . . . . . . . . . . . . 2142
13.1.10.1.2 Least Squares Method (LSM) . . . . . . . . . . . . 2143
13.1.10.1.3 Univariate Regression Variance Analysis . . . . . . 2146
13.1.10.2 Univariate linear regression Gaussian Model . . . . . . . . . 2151
13.1.10.2.1 Pearson Correlation Coefﬁcient Test . . . . . . . . . 2160
13.1.10.2.2 Conﬁdence interval of predicted values . . . . . . . 2162
13.1.10.3 Linear univariate regression forced through the origin . . . . 2167
13.1.10.4 Multiple linear regression Gaussian Model . . . . . . . . . . 2168
13.1.10.5 Polynomial regression . . . . . . . . . . . . . . . . . . . . . 2176
13.1.10.6 Logistic Regressions (LOGIT) . . . . . . . . . . . . . . . . . 2178
13.1.10.6.1 Binomial Logistic Regression . . . . . . . . . . . . 2178
13.1.10.6.2 ROC and Lift curves . . . . . . . . . . . . . . . . . 2187
13.1.11Interpolation Techniques . . . . . . . . . . . . . . . . . . . . . . . . . 2202
13.1.11.1 Bezier Curves (B-Splines) . . . . . . . . . . . . . . . . . . . 2202
13.1.11.2 Euler Method . . . . . . . . . . . . . . . . . . . . . . . . . . 2209
13.1.11.3 Polynomial of collocation . . . . . . . . . . . . . . . . . . . 2211
13.1.11.4 Lagrange polynomial interpolation method . . . . . . . . . . 2214
13.1.12Roots search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2216
13.1.12.1 Proportional parts methods . . . . . . . . . . . . . . . . . . . 2216
13.1.12.2 Bisection method . . . . . . . . . . . . . . . . . . . . . . . . 2218
13.1.12.3 Secant method (Regula Falsi or False Position) . . . . . . . . 2220
13.1.12.4 Newton’s method . . . . . . . . . . . . . . . . . . . . . . . . 2222
13.1.13Numerical Differentiation . . . . . . . . . . . . . . . . . . . . . . . . . 2227
13.1.14Numerical Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . 2229
13.1.14.1 Rectangles method . . . . . . . . . . . . . . . . . . . . . . . 2230
13.1.14.2 Trapezoidal method . . . . . . . . . . . . . . . . . . . . . . 2231
13.1.15Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2232
13.1.15.1 Linear programming (Linear Optimization) . . . . . . . . . . 2233
13.1.15.1.1 Graphical LP resolution . . . . . . . . . . . . . . . 2236
13.1.15.1.2 Algebraic LP resolution . . . . . . . . . . . . . . . 2237
13.1.15.1.3 Simplex algorithm LP resolution . . . . . . . . . . 2238
13.1.16Monte Carlo Simulations . . . . . . . . . . . . . . . . . . . . . . . . . 2238
13.1.16.1 Inverse Transform Sampling . . . . . . . . . . . . . . . . . . 2239
xxiv

Draft
13.2 Fractals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2241
13.2.1 IFS Fractals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2243
13.2.1.1 Fractals Metric Space . . . . . . . . . . . . . . . . . . . . . 2258
13.2.2 Fractals Visualization . . . . . . . . . . . . . . . . . . . . . . . . . . . 2262
13.3 Logical Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2263
13.3.1 Strict Logic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2263
13.3.1.1 Boolean Algebra . . . . . . . . . . . . . . . . . . . . . . . . 2264
13.3.1.2 Logical Functions . . . . . . . . . . . . . . . . . . . . . . . 2271
13.4 Error-Correcting Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2273
13.4.1 CheckSum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2276
13.4.2 Encoders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2277
13.5 Automata Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2282
13.5.1 Von Neumann machine . . . . . . . . . . . . . . . . . . . . . . . . . . 2284
13.6 Cryptography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2285
13.6.1 Cryptographic systems . . . . . . . . . . . . . . . . . . . . . . . . . . 2286
13.6.1.1 Kerckhoffs’ principle . . . . . . . . . . . . . . . . . . . . . . 2290
13.6.2 Traps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2291
13.6.3 Secret-key encryption system . . . . . . . . . . . . . . . . . . . . . . . 2292
13.6.3.1 Feistel Schemes . . . . . . . . . . . . . . . . . . . . . . . . 2294
13.7 Quantum Computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2296
13.7.1 Schrödinger’s Cat superposition . . . . . . . . . . . . . . . . . . . . . 2298
13.7.2 Photon polarization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2299
14 Social Sciences 2302
14.1 Population Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2304
14.1.1 Birth rate and mortality tables (biometric features) . . . . . . . . . . . 2304
14.1.1.1 Population Renevewal . . . . . . . . . . . . . . . . . . . . . 2312
14.1.2 Population Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2314
14.1.2.1 Exponential model . . . . . . . . . . . . . . . . . . . . . . . 2314
14.1.2.2 Deterministic Logistic Model (Verlhust) . . . . . . . . . . . . 2317
14.1.2.3 Chaotic Logistic Model . . . . . . . . . . . . . . . . . . . . 2321
14.1.2.3.1 Feigenbaum’s Bifurcation Diagram . . . . . . . . . 2325
14.1.2.4 Malthusian Growth Law . . . . . . . . . . . . . . . . . . . . 2329
14.1.2.5 Leslie model . . . . . . . . . . . . . . . . . . . . . . . . . . 2330
14.1.2.6 SIR Model for Spread of Disease . . . . . . . . . . . . . . . 2332
14.1.2.7 Lotka–Volterra predator–prey model . . . . . . . . . . . . . . 2335
14.1.3 Schaefer’s Optimal capture model . . . . . . . . . . . . . . . . . . . . 2344
14.1.4 Hardy-Weinberg model . . . . . . . . . . . . . . . . . . . . . . . . . . 2346
14.1.5 Mendel’s law . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2352
14.1.6 Growth rate with temperature . . . . . . . . . . . . . . . . . . . . . . . 2353
14.2 Game and Decision Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2354
14.2.1 Utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2357
14.2.1.1 Pareto Optimum . . . . . . . . . . . . . . . . . . . . . . . . 2357
14.2.1.2 Nash Equilibrium . . . . . . . . . . . . . . . . . . . . . . . 2358
14.2.2 Games Representations . . . . . . . . . . . . . . . . . . . . . . . . . . 2361
14.2.2.1 Extensive representation of a Game . . . . . . . . . . . . . . 2363
14.2.2.2 Extensive representation of a Decision . . . . . . . . . . . . 2364
14.2.2.2.1 Real Options . . . . . . . . . . . . . . . . . . . . . 2371
xxv

Draft
14.2.2.3 Normal representation of a Game . . . . . . . . . . . . . . . 2373
14.2.2.3.1 Repetitive Games . . . . . . . . . . . . . . . . . . 2380
14.2.2.4 Set representation of a Game . . . . . . . . . . . . . . . . . . 2382
14.2.2.5 Graphical representation of a Game . . . . . . . . . . . . . . 2385
14.2.3 Expected Utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2389
14.2.3.1 Hurwitz Criteria . . . . . . . . . . . . . . . . . . . . . . . . 2389
14.2.3.2 Laplace Criteria . . . . . . . . . . . . . . . . . . . . . . . . 2392
14.2.4 Evolutionary Game . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2393
14.2.5 Multi-Criteria Decision Making (MCDM) . . . . . . . . . . . . . . . . 2393
14.2.5.1 Analytic Hierarchy Process (AHP) . . . . . . . . . . . . . . 2393
14.3 Economy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2395
14.3.1 Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2395
14.3.1.1 Microeconomics . . . . . . . . . . . . . . . . . . . . . . . . 2396
14.3.1.1.1 Average & Marginal Cost . . . . . . . . . . . . . . 2401
14.3.1.2 Macroeconomics . . . . . . . . . . . . . . . . . . . . . . . . 2406
14.3.1.2.1 Cobb-Douglas Model . . . . . . . . . . . . . . . . 2407
14.3.2 Monetary Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2412
14.3.2.1 Walras’ law . . . . . . . . . . . . . . . . . . . . . . . . . . . 2414
14.3.3 Supply and Demand Theory . . . . . . . . . . . . . . . . . . . . . . . 2420
14.3.3.1 Expected utility theory . . . . . . . . . . . . . . . . . . . . . 2420
14.3.4 Net gain/loss opposite feedback model . . . . . . . . . . . . . . . . . . 2426
14.3.5 Capitalization and Actuarial . . . . . . . . . . . . . . . . . . . . . . . 2434
14.3.5.1 Dates Intervals . . . . . . . . . . . . . . . . . . . . . . . . . 2436
14.3.5.2 Rates Equivalence . . . . . . . . . . . . . . . . . . . . . . . 2439
14.3.5.3 Simple Interest . . . . . . . . . . . . . . . . . . . . . . . . . 2440
14.3.5.3.1 Discounts . . . . . . . . . . . . . . . . . . . . . . . 2441
14.3.5.4 Compound Interest . . . . . . . . . . . . . . . . . . . . . . . 2443
14.3.5.5 Continuous Interest . . . . . . . . . . . . . . . . . . . . . . . 2445
14.3.6 Progressive interest (annuities) . . . . . . . . . . . . . . . . . . . . . . 2446
14.3.6.1 Postnumerando annuities . . . . . . . . . . . . . . . . . . . . 2447
14.3.6.2 Praenumerando annuities . . . . . . . . . . . . . . . . . . . 2450
14.3.7 Rounding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2452
14.3.8 Loans Amortization/Repayments . . . . . . . . . . . . . . . . . . . . . 2454
14.3.8.1 Fixed-Term Loan . . . . . . . . . . . . . . . . . . . . . . . . 2456
14.3.8.2 Loan with constant amortization . . . . . . . . . . . . . . . . 2457
14.3.8.3 Loan with constant annuity . . . . . . . . . . . . . . . . . . 2457
14.3.9 Modern Portfolio Theory . . . . . . . . . . . . . . . . . . . . . . . . . 2460
14.3.9.1 No Arbitrage Opportunity (N.A.O.) . . . . . . . . . . . . . . 2463
14.3.9.2 Portfolios . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2466
14.3.9.2.1 Stocks (shares of stocks)/Equities . . . . . . . . . . 2471
14.3.9.2.1.1 Dividend Yield . . . . . . . . . . . . . . . . 2473
14.3.9.2.1.2 Shares Benchmark Indices . . . . . . . . . . 2475
14.3.9.2.1.3 Durand Model . . . . . . . . . . . . . . . . 2476
14.3.9.2.2 Obligations (Bonds) . . . . . . . . . . . . . . . . . 2479
14.3.9.2.3 Warrants . . . . . . . . . . . . . . . . . . . . . . . 2493
14.3.9.2.4 Futures & Forwards . . . . . . . . . . . . . . . . . 2497
14.3.9.2.4.1 Futures & Forwards naive pricing . . . . . . 2502
14.3.9.2.4.2 Futures & Forwards commodity hedging . . 2506
xxvi

Draft
14.3.9.2.4.3 Options . . . . . . . . . . . . . . . . . . . . 2512
14.3.10Time Series Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 2524
14.4 Quantitative Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2527
14.4.1 Corporate Finance Management . . . . . . . . . . . . . . . . . . . . . 2530
14.4.1.1 Basic Accounting Equation . . . . . . . . . . . . . . . . . . 2531
14.4.1.2 Ratio Analysis . . . . . . . . . . . . . . . . . . . . . . . . . 2532
14.4.1.2.1 Short term solvency or liquidity measure . . . . . . 2533
14.4.1.2.2 Long term solvency or Liquidity measure . . . . . . 2534
14.4.1.2.3 Profability Measures . . . . . . . . . . . . . . . . . 2535
14.4.1.2.4 Growth Rate . . . . . . . . . . . . . . . . . . . . . 2536
14.4.1.2.5 Asset management or turnovers measures . . . . . . 2538
14.4.1.2.6 Market Value Measures . . . . . . . . . . . . . . . 2538
14.4.1.3 Weighted average cost of capital (WACC) . . . . . . . . . . . 2539
14.4.1.4 Break-even Point Analysis (BEPA) . . . . . . . . . . . . . . 2541
14.4.1.5 Investment Strategies . . . . . . . . . . . . . . . . . . . . . . 2543
14.4.1.6 Capital Goods . . . . . . . . . . . . . . . . . . . . . . . . . 2544
14.4.2 Linear Amortization . . . . . . . . . . . . . . . . . . . . . . . . . . . 2544
14.4.2.1 Wages model . . . . . . . . . . . . . . . . . . . . . . . . . . 2545
14.4.3 Project Management . . . . . . . . . . . . . . . . . . . . . . . . . . . 2547
14.4.3.1 Probabilistic PERT . . . . . . . . . . . . . . . . . . . . . . . 2547
14.4.4 Lean Management (Six Sigma Process) . . . . . . . . . . . . . . . . . 2555
14.4.4.1 Pareto Analysis . . . . . . . . . . . . . . . . . . . . . . . . . 2558
14.4.4.1.1 Gini Index . . . . . . . . . . . . . . . . . . . . . . 2562
14.4.4.2 Weighted Ishikawa Diagram . . . . . . . . . . . . . . . . . . 2565
14.4.5 Supply Chain Management . . . . . . . . . . . . . . . . . . . . . . . . 2569
14.4.5.1 Supply Chain Management in uncertain future . . . . . . . . 2571
14.4.5.2 Optimal initial stock management with zero rotation . . . . . 2573
14.4.5.3 Wilson’s Models . . . . . . . . . . . . . . . . . . . . . . . . 2580
14.4.5.3.1 Wilson’s model with resupply . . . . . . . . . . . . 2586
14.4.5.3.2 Wilson’s model without resupply . . . . . . . . . . 2594
14.4.5.3.3 Wilson’s model with resupply and break-up . . . . . 2596
14.4.6 Sensitivity Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2601
14.4.6.1 Direct Bias Method . . . . . . . . . . . . . . . . . . . . . . 2603
14.4.6.2 Correlation Method . . . . . . . . . . . . . . . . . . . . . . 2607
14.5 Music Maths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2612
14.5.1 Longitudinal Sound Waves . . . . . . . . . . . . . . . . . . . . . . . . 2612
15 Engineering 2617
15.1 Marine & Weather Engineering . . . . . . . . . . . . . . . . . . . . . . . . . . 2618
15.1.1 Visual horizon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2618
15.2 Mechanical Engineering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2621
15.2.1 Gears . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2621
15.2.1.1 Transmission ratios . . . . . . . . . . . . . . . . . . . . . . . 2624
15.2.1.2 Gears association . . . . . . . . . . . . . . . . . . . . . . . . 2627
15.2.1.2.1 Odd/Even Gear "problem" . . . . . . . . . . . . . . 2631
15.2.1.3 Type of Gears . . . . . . . . . . . . . . . . . . . . . . . . . 2633
15.2.2 Strength of materials . . . . . . . . . . . . . . . . . . . . . . . . . . . 2636
15.3 Electrical Engineering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2638
xxvii

Draft
15.4 Civil Engineering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2640
15.4.1 Static . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2641
15.4.2 Pulleys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2642
15.4.2.1 Windlass . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2648
15.4.3 Cornu spiral . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2650
15.5 Aerospace Engineering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2652
15.5.1 Cosmological Speeds . . . . . . . . . . . . . . . . . . . . . . . . . . . 2653
15.6 Software Engineering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2655
15.6.1 Google PageRank algorithm . . . . . . . . . . . . . . . . . . . . . . . 2655
15.6.1.1 Weighted Count . . . . . . . . . . . . . . . . . . . . . . . . 2657
15.7 Industrial Engineering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2659
15.7.1 Six Sigma . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2660
15.7.1.1 Quality Control . . . . . . . . . . . . . . . . . . . . . . . . . 2662
15.7.1.2 Defaults/Errors . . . . . . . . . . . . . . . . . . . . . . . . . 2663
15.7.1.3 Capability Indices . . . . . . . . . . . . . . . . . . . . . . . 2668
15.7.1.4 Quality Levels . . . . . . . . . . . . . . . . . . . . . . . . . 2680
15.7.2 Taguchi Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2693
15.7.3 Preventive Maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . 2699
15.7.3.1 Planned Obsolescence . . . . . . . . . . . . . . . . . . . . . 2700
15.7.3.2 Reliability Empirical Estimators . . . . . . . . . . . . . . . . 2700
15.7.3.3 Weibull Distribution . . . . . . . . . . . . . . . . . . . . . . 2715
15.7.3.4 Topology of Systems . . . . . . . . . . . . . . . . . . . . . . 2722
15.7.3.4.1 Fault Tree Analysis . . . . . . . . . . . . . . . . . 2733
15.7.3.5 Maximum Likelihood for failure rate determination of samples2735
15.7.3.6 Kaplan-Meier Survival Rate . . . . . . . . . . . . . . . . . . 2737
15.7.3.7 ABC Method . . . . . . . . . . . . . . . . . . . . . . . . . . 2742
15.7.4 Design of Experiments (DoE) . . . . . . . . . . . . . . . . . . . . . . 2747
15.7.4.1 Full Factorial Designs . . . . . . . . . . . . . . . . . . . . . 2755
15.7.4.2 Fractional Factorial Designs . . . . . . . . . . . . . . . . . . 2766
15.7.4.3 Taguchi Designs and Nomenclature . . . . . . . . . . . . . . 2774
16 Epilogue 2777
17 Biographies 2779
A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2780
B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2782
C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2789
D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2796
E . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2800
F . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2803
G . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2808
H . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2813
I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2821
J . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2821
K . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2823
L . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2826
M . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2834
N . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2842
O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2846
xxviii

Draft
P . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2847
R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2852
S . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2853
T . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2861
V . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2864
W . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2865
Y . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2868
Z . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2870
18 Chronology 2872
19 Humour 2902
19.1 Situations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2903
19.2 Mathematics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2915
19.3 Physics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2933
19.4 Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2949
19.5 Chemistry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2952
19.6 Engineering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2958
19.7 Computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2965
19.8 Social Sciences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2974
20 Links 2980
20.1 Exact Sciences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2981
20.2 Publishing/Magazines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2982
20.3 Associations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2984
20.4 Jobs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2985
20.5 Television/Radio . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2985
20.6 Other sciences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2986
20.7 Softwares . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2986
21 Quotes 2990
22 Change Log 2995
23 Nomenclature 3012
24 About the Redactor 3014
List of Figures 3018
List of Tables 3038
Bibliography 3042
Index 3046
xxix

Draft
1Warnings
Contents
1.1 Impressum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1.1 Use of content . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1.2 How to use this book . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1.3 Data Protection . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.1.4 Use of data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.1.5 Data transmission . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.1.6 Agreement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2 License . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.2.1 Preamble . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.2.2 Applicability and Deﬁnitions . . . . . . . . . . . . . . . . . . . . . 8
1.2.3 Verbatim Copying . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.2.4 Copying in Quantity . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.2.5 Modiﬁcations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.2.6 Combining Documents . . . . . . . . . . . . . . . . . . . . . . . . 11
1.2.7 Collections of Documents . . . . . . . . . . . . . . . . . . . . . . . 12
1.2.8 Aggregation with independant Works . . . . . . . . . . . . . . . . . 12
1.2.9 Translation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.2.10 Termination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.2.11 Future revisions of this License . . . . . . . . . . . . . . . . . . . . 13
1.3 Only in non-free release . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1

Draft
1. Warnings Vincent ISOZ [EAME v3.0-2013]
1.1 Impressum
1.1.1 Use of content
The contents of this book are elaborated by a development process by which volunteers reach
a consensus. This process that brings together volunteers, research also the point of view of
people interested in the topics of this book. The person in charge of this book administers
the process and establishes rules to promote fairness in the consensus approach. It is also
responsible for drafting the text, sometime for testing/evaluating or independently verifying the
accuracy or completeness of the presented information.
We decline no responsibility for any injury, damage or any other kind, special, incidental, con-
sequential or compensatory, arising from the publication, application or reliance on the content
of this book. We make no express or implied warranty on the accuracy or completeness of any
information published in this book, and do not guarantee that the information contained in this
book meet any specific need or goal of the reader. We do not guarantee the performance of
products or services of one manufacturer or vendor solely by virtue of this book content.
The technical descriptions, procedures, and computer programs in this book have been devel-
oped without care, therefore they are provide without warranty of any kind. We make also no
warranties that the equations, programs, and procedures in this books or its associated software
are free of error, or are consistent with any particular standard of merchantability, or will meet
your requirements for any particular application. They should not be relied upon for solving a
problem whose incorrect solution could result in injury to a person or loss of property. Any use
of the content of this book as at the reader’s own risk. The authors, redactors, and publisher dis-
claim all liability for direct, incidental, or consequent damages resulting form use of the content
of this book or the associated software.
By publishing texts, it is not the intention of this book to provide services on behalf of any
person or entity or performing any task to be accomplished by any person or entity for the
benefit of a third party. Anyone using this book should rely on its own independent judgment
or, where that is appropriate, seek the advice of a qualified expert to determine how to exercise
reasonable care under all circumstances. The information and standards on the topics covered
by this book may be available from other sources that the reader may wish to visit in search of
points of view or additional information not covered by the contents of this book.
We have no power in order to enforce compliance with the contents of this book, and we do not
undertake to monitor or enforce such compliance. We have no certification, testing or inspection
activity of products, designs or installations for safety or health of persons and property. Any
certification or other statement of compliance regarding information relating to health or safety
of persons and property, mentioned in this book, cannot possibly be attributed to the content
of this book and remains under the responsibility of the certification center or the concerned
reporter.
2/3049 Sciences.ch

Draft
Vincent ISOZ [EAME v3.0-2013] 1. Warnings
1.1.2 How to use this book
At the university level, this book can be used for a Ph.D., graduate level or advanced under-
graduate level seminar in many exact and pure sciences fields. The seminars where we use this
material is part of Scientific Evolution Sàrl program, where the trainees typically already have
taken undergraduate or graduate courses in their respective specialization. In reality this books
also aims to cover the full Kindergarten to PhD curriculum.
Because the methods of Applied Mathematics are learned by practice and experience, we view
a seminar on Applied Mathematics as a learning-by-doing (project oriented) seminar. We struc-
ture our mathematical modelling seminars around a set of problems that require the trainee
to construct models that help with planning and decision making. The imperative is that the
models should be consistent with the theory and back-tested. To fulfill this imperative, it is
necessary for the trainee to combine mathematical theory with modeling. The result is that the
trainee learns the theory, and more importantly, learns how that theory is applied and combined
in the real world. The ability to criticize and identify limitations of dangerous mathematical
tools is the most valuable feature of our seminars.
The problems with solutions in this book provide the opportunity to apply the text material to a
comprehensive set of fairly realistic situations. By the end of the seminars the trainees will have
enhanced their skills and knowledge of the most important theoretical and computing tools.
These are valuable skills that are in demand by the businesses at the highest levels.
It is very difficult to cover all the material in this book in a semester. It takes a lot of time to
explain the concepts to the trainees. The reader is encouraged to pick and choose which topics
will be covered during the term. It is not necessary strictly necessary to cover them in sequence
but it can help in a significant way?
In a nutshell, this book offers you a wide variety of topics that are amenable to modeling. All
are practical.
1.1.2.1 Companion eBooks, Forum and Quiz/Flashcards
There are some free companion books and tools in French and English written by Vincent ISOZ
& Daname KOLANI for the people that want to put in practice the theory presented in this book.
Here is the list:
• MATLAB™ in English (free sample of 1,059 pages, complete eBook with data 499$):
http://www.sciences.ch/dwnldbl/divers/Matlab.pdf
• Maple in French (free 99 pages):
http://www.sciences.ch/dwnldbl/divers/Maple.pdf
• R in French (free sample of 1,570 pages, complete eBook with data 499$):
http://www.sciences.ch/dwnldbl/divers/R.pdf
• Minitab in French (free sample of 1,060 pages, complete eBook with data 99$ ):
Sciences.ch 3/3049

Draft
http://www.sciences.ch/dwnldbl/divers/Minitab.pdf
• Scientific Linux installation & Configuration (199 pages completely free):
http://www.sciences.ch/dwnldbl/divers/ScientificLinux.pdf
and some available Quiz and Flash Cards in French and English to challenge yourself with the
rest of the world:
• MATLAB™ Basics L1 Challenge level in French (100 questions)
httpXGGwwwFs™ientifi™EevolutionF™omGq™mGst—rt•sessionG—UQTRU™fQ˜G
• Astronomy/Astrophysics H1 Challenge level in English (100 questions):
httpXGGwwwFs™ientifi™EevolutionF™omGq™mGst—rt•sessionGffdHVIHf—HG
• Greek Letter Flashcards (48 cards):
httpXGGwwwFs™ientifi™EevolutionF™omGq™mGfrGst—rt•sessionGTdWfIfefWHG
• Common Derivatives Flashcards (29 cards):
httpXGGwwwFs™ientifi™EevolutionF™omGq™mGfrGst—rt•sessionG™IS—RHfP™RG
• Common Primitives Flashcards (60 cards):
httpXGGwwwFs™ientifi™EevolutionF™omGq™mGfrGst—rt•sessionG™™f™PHfdefG
• LATEX L3 Challenge level in French (100 questions):
httpXGGwwwFs™ientifi™EevolutionF™omGq™mGfrGst—rt•sessionGffIeIdI˜WIG
• R Software 3.1.2 L3 Challenge level in French (100 questions):
httpXGGwwwFs™ientifi™EevolutionF™omGq™mGfrGst—rt•sessionGP—Tf™—URUQG
• C++ L3 Challenge level in French (100 questions):
httpXGGwwwFs™ientifi™EevolutionF™omGq™mGfrGst—rt•sessionGeHQI™eR˜RQG
But about the forum (as I don’t like to recreate the wheel), go through this link for any discus-
sions about the content of this book:
httpsXGGwwwFphysi™sforumsF™om
As for this book, the companion books above are only samples of the complete one. The full
version with perpetual free updates are available for the price of $ 299.- each and for $ 499.-
you get the exercise files and LATEX sources (for information on purchase you can simply send
me an email).
4/3049 Sciences.ch

Draft
Because this book mainly focus on mathematical aspect of physical phenomena we can only
strongly recommend to the reader an another free book that is in our point of view actually the
best one that focus on the popular science aspect of the subjects that we will cover:
Motion Mountain by Dr. Christoph Schiller: httpXGGwwwFmotionmount—inFnet
Sciences.ch 5/3049

Draft
1.1.3 Data Protection
When looking at information on the Internet companion site (Sciences.ch), some data are au-
tomatically saved. We try to save as less as possible data and as brief as possible. Wherever
we can, we ave only anonymous data. We undertake to process the data you send us personally
with the utmost diligence.
However, your IP address and the source page that takes you on Sciences.ch and the associated
keywords, are freely available to everybody here for the current month. After which detailed
data are destroyed. You can object at any time in the publication of your data by contacting us.
1.1.4 Use of data
Your data are only used for sending the Sciences.ch newsletter. Communication of personal
data (except the e-mail address, title and name) is optional. When registering for the newsletter,
you can of course specify an alternate address and/or a ﬁctitious name.
1.1.5 Data transmission
We will never sell or commercialize the data of our customers or interested parties and will
never affects the rights of the person. In addition, we will not rent mailing lists and will not
send you advertising from third parties or on our behalf.
1.1.6 Agreement
When you provide us personal information, you authorize us to save them and use them within
the meaning of the Swiss Federal Law on Data Protection. If you ask us not to send you emails,
we are obliged, in your interest, save your e-mail in an internal negative list.
6/3049 Sciences.ch

Draft
1.2 License
The entire contents of this book is subject to the GNU Free Documentation License, which
means:
• that everyone has the right to freely use the texts for non-commercial usage (Google Ads
or any equivalent being considered as a commercial usage!)
• that any person is authorized to broadcast items for non-commercial usage (Google Ads
or any equivalent being considered as a commercial usage!)
• that anyone can freely edit the texts for non-commmercial usage (Google Ads or any
equivalent being considered as a commercial usage!)
and bla bla bla...
in accordance with the license described below:
Version 1.1, March 2000
Copyright (C) 2000 Free Software Foundation, Inc. 59 Temple Place, Suite 330, Boston, MA
02111-1307 USA Everyone is permitted to copy and distribute verbatim copies of this license
document, but changing it is not allowed.
1.2.1 Preamble
The purpose of this License is to make a manual, textbook, or other written document "free" in
the sense of freedom: to assure everyone the effective freedom to copy and redistribute it, with
or without modifying it only a non-commercial purpose. Secondarily, this License preserves
for the author and publisher a way to get credit for their work, while not being considered
responsible for modiﬁcations made by others.
This License is a kind of "copyleft", which means that derivative works of the document must
themselves be free in the same sense. It complements the GNU General Public License, which
is a copyleft license designed for free software.
We have designed this License in order to use it for manuals for free software, because free
software needs free documentation: a free program should come with manuals providing the
same freedoms that the software does. But this License is not limited to software manuals; it
can be used for any textual work, regardless of subject matter or whether it is published as a
printed book. We recommend this License principally for works whose purpose is instruction
or reference.
Sciences.ch 7/3049

Draft
1.2.2 Applicability and Definitions
This License applies to any manual or other work that contains a notice placed by the copyright
holder saying it can be distributed under the terms of this License. The "Document", below,
refers to any such manual or work. Any member of the public is a licensee, and is addressed as
"you".
A "Modified Version" of the Document means any work containing the Document or a portion
of it, either copied verbatim, or with modifications and/or translated into another language.
A "Secondary Section" is a named appendix or a front-matter section of the Document that deals
exclusively with the relationship of the publishers or authors of the Document to the Document’s
overall subject (or to related matters) and contains nothing that could fall directly within that
overall subject. (For example, if the Document is in part a textbook of mathematics, a Secondary
Section may not explain any mathematics.) The relationship could be a matter of historical
connection with the subject or with related matters, or of legal, commercial, philosophical,
ethical or political position regarding them.
The "Invariant Sections" are certain Secondary Sections whose titles are designated, as being
those of Invariant Sections, in the notice that says that the Document is released under this
License.
The "Cover Texts" are certain short passages of text that are listed, as Front-Cover Texts or
Back-Cover Texts, in the notice that says that the Document is released under this License.
A "Transparent" copy of the Document means a machine-readable copy, represented in a for-
mat whose specification is available to the general public, whose contents can be viewed and
edited directly and straightforwardly with generic text editors or (for images composed of pix-
els) generic paint programs or (for drawings) some widely available drawing editor, and that
is suitable for input to text formatters or for automatic translation to a variety of formats suit-
able for input to text formatters. A copy made in an otherwise Transparent file format whose
markup has been designed to thwart or discourage subsequent modification by readers is not
Transparent. A copy that is not "Transparent" is named "Opaque".
Examples of suitable formats for Transparent copies include plain ASCII without markup, Tex-
info input format, LaTeX input format, SGML or XML using a publicly available DTD, and
standard-conforming simple HTML designed for human modification. Opaque formats include
PostScript, PDF, proprietary formats that can be read and edited only by proprietary word pro-
cessors, SGML or XML for which the DTD and/or processing tools are not generally available,
and the machine-generated HTML produced by some word processors for output purposes only.
The "Title Page" means, for a printed book, the title page itself, plus such following pages as
are needed to hold, legibly, the material this License requires to appear in the title page. For
works in formats which do not have any title page as such, "Title Page" means the text near the
most prominent appearance of the work’s title, preceding the beginning of the body of the text.
8/3049 Sciences.ch

Draft
1.2.3 Verbatim Copying
You may copy and distribute the Document in any medium, noncommercially, provided that
this License, the copyright notices, and the license notice saying this License applies to the
Document are reproduced in all copies, and that you add no other conditions whatsoever to
those of this License. You may not use technical measures to obstruct or control the reading or
further copying of the copies you make or distribute. However, you may accept compensation
in exchange for copies. If you distribute a large enough number of copies you must also follow
the conditions in section 3.
You may also lend copies, under the same conditions stated above, and you may publicly display
copies.
1.2.4 Copying in Quantity
If you publish printed copies of the Document numbering more than 100, and the Document’s
license notice requires Cover Texts, you must enclose the copies in covers that carry, clearly
and legibly, all these Cover Texts: Front-Cover Texts on the front cover, and Back-Cover Texts
on the back cover. Both covers must also clearly and legibly identify you as the publisher
of these copies. The front cover must present the full title with all words of the title equally
prominent and visible. You may add other material on the covers in addition. Copying with
changes limited to the covers, as long as they preserve the title of the Document and satisfy
these conditions, can be treated as verbatim copying in other respects.
If the required texts for either cover are too voluminous to fit legibly, you should put the first
ones listed (as many as fit reasonably) on the actual cover, and continue the rest onto adjacent
pages.
If you publish or distribute Opaque copies of the Document numbering more than 100, you
must either include a machine-readable Transparent copy along with each Opaque copy, or
state in or with each Opaque copy a publicly-accessible computer-network location containing a
complete Transparent copy of the Document, free of added material, which the general network-
using public has access to download anonymously at no charge using public-standard network
protocols. If you use the latter option, you must take reasonably prudent steps, when you begin
distribution of Opaque copies in quantity, to ensure that this Transparent copy will remain thus
accessible at the stated location until at least one year after the last time you distribute an Opaque
copy (directly or through your agents or retailers) of that edition to the public.
It is requested, but not required, that you contact the authors of the Document well before
redistributing any large number of copies, to give them a chance to provide you with an updated
version of the Document.
Sciences.ch 9/3049

Draft
1.2.5 Modifications
You may copy and distribute a Modified Version of the Document under the conditions of
sections 2 and 3 above, provided that you release the Modified Version under precisely this
License, with the Modified Version filling the role of the Document, thus licensing distribution
and modification of the Modified Version to whoever possesses a copy of it. In addition, you
must do these things in the Modified Version:
• Use in the Title Page (and on the covers, if any) a title distinct from that of the Document,
and from those of previous versions (which should, if there were any, be listed in the
History section of the Document). You may use the same title as a previous version if the
original publisher of that version gives permission.
• List on the Title Page, as authors, one or more persons or entities responsible for au-
thorship of the modifications in the Modified Version, together with at least five of the
principal authors of the Document (all of its principal authors, if it has less than five).
• State on the Title page the name of the publisher of the Modified Version, as the publisher.
• Preserve all the copyright notices of the Document.
• Add an appropriate copyright notice for your modifications adjacent to the other copyright
notices.
• Include, immediately after the copyright notices, a license notice giving the public per-
mission to use the Modified Version under the terms of this License, in the form shown
in the Addendum below.
• Preserve in that license notice the full lists of Invariant Sections and required Cover Texts
given in the Document’s license notice.
• Include an unaltered copy of this License.
• Preserve the section entitled "History", and its title, and add to it an item stating at least
the title, year, new authors, and publisher of the Modified Version as given on the Title
Page. If there is no section entitled "History" in the Document, create one stating the title,
year, authors, and publisher of the Document as given on its Title Page, then add an item
describing the Modified Version as stated in the previous sentence.
• Preserve the network location, if any, given in the Document for public access to a Trans-
parent copy of the Document, and likewise the network locations given in the Document
for previous versions it was based on. These may be placed in the "History" section. You
may omit a network location for a work that was published at least four years before the
Document itself, or if the original publisher of the version it refers to gives permission.
• In any section entitled "Acknowledgements" or "Dedications", preserve the section’s title,
and preserve in the section all the substance and tone of each of the contributor acknowl-
edgements and/or dedications given therein.
• Preserve all the Invariant Sections of the Document, unaltered in their text and in their
titles. Section numbers or the equivalent are not considered part of the section titles.
10/3049 Sciences.ch

Draft
• Delete any section entitled "Endorsements". Such a section may not be included in the
Modified Version.
• Do not retitle any existing section as "Endorsements" or to conflict in title with any In-
variant Section.
• If the Modified Version includes new front-matter sections or appendices that qualify as
Secondary Sections and contain no material copied from the Document, you may at your
option designate some or all of these sections as invariant. To do this, add their titles to
the list of Invariant Sections in the Modified Version’s license notice. These titles must
be distinct from any other section titles.
• You may add a section entitled "Endorsements", provided it contains nothing but endorse-
ments of your Modified Version by various parties–for example, statements of peer review
or that the text has been approved by an organization as the authoritative definition of a
standard.
• You may add a passage of up to five words as a Front-Cover Text, and a passage of up
to 25 words as a Back-Cover Text, to the end of the list of Cover Texts in the Modified
Version. Only one passage of Front-Cover Text and one of Back-Cover Text may be added
by (or through arrangements made by) any one entity. If the Document already includes
a cover text for the same cover, previously added by you or by arrangement made by the
same entity you are acting on behalf of, you may not add another; but you may replace
the old one, on explicit permission from the previous publisher that added the old one.
• The author(s) and publisher(s) of the Document do not by this License give permission
to use their names for publicity for or to assert or imply endorsement of any Modified
Version.
1.2.6 Combining Documents
You may combine the Document with other documents released under this License, under the
terms defined in section 4 above for modified versions, provided that you include in the combi-
nation all of the Invariant Sections of all of the original documents, unmodified, and list them
all as Invariant Sections of your combined work in its license notice.
The combined work need only contain one copy of this License, and multiple identical Invariant
Sections may be replaced with a single copy. If there are multiple Invariant Sections with the
same name but different contents, make the title of each such section unique by adding at the
end of it, in parentheses, the name of the original author or publisher of that section if known,
or else a unique number. Make the same adjustment to the section titles in the list of Invariant
Sections in the license notice of the combined work.
In the combination, you must combine any sections entitled "History" in the various original
documents, forming one section entitled "History"; likewise combine any sections entitled "Ac-
knowledgements", and any sections entitled "Dedications". You must delete all sections entitled
"Endorsements."
Sciences.ch 11/3049

Draft
1.2.7 Collections of Documents
You may make a collection consisting of the Document and other documents released under
this License, and replace the individual copies of this License in the various documents with a
single copy that is included in the collection, provided that you follow the rules of this License
for verbatim copying of each of the documents in all other respects.
You may extract a single document from such a collection, and distribute it individually under
this License, provided you insert a copy of this License into the extracted document, and follow
this License in all other respects regarding verbatim copying of that document.
1.2.8 Aggregation with independant Works
A compilation of the Document or its derivatives with other separate and independent docu-
ments or works, in or on a volume of a storage or distribution medium, does not as a whole
count as a Modiﬁed Version of the Document, provided no compilation copyright is claimed for
the compilation. Such a compilation is named an "aggregate", and this License does not apply
to the other self-contained works thus compiled with the Document, on account of their being
thus compiled, if they are not themselves derivative works of the Document.
If the Cover Text requirement of section 3 is applicable to these copies of the Document, then
if the Document is less than one quarter of the entire aggregate, the Document’s Cover Texts
may be placed on covers that surround only the Document within the aggregate. Otherwise they
must appear on covers around the whole aggregate.
1.2.9 Translation
Translation is considered a kind of modiﬁcation, so you may distribute translations of the Doc-
ument under the terms of the corresponding section about transformation. Replacing Invariant
Sections with translations requires special permission from their copyright holders, but you may
include translations of some or all Invariant Sections in addition to the original versions of these
Invariant Sections. You may include a translation of this License provided that you also include
the original English version of this License. In case of a disagreement between the translation
and the original English version of this License, the original English version will prevail.
1.2.10 Termination
You may not copy, modify, sublicense, or distribute the Document except as expressly provided
for under this License. Any other attempt to copy, modify, sublicense or distribute the Docu-
ment is void, and will automatically terminate your rights under this License. However, parties
who have received copies, or rights, from you under this License will not have their licenses
terminated so long as such parties remain in full compliance.
12/3049 Sciences.ch

Draft
1.2.11 Future revisions of this License
The Free Software Foundation may publish new, revised versions of the GNU Free Documenta-
tion License from time to time. Such new versions will be similar in spirit to the present version,
but may differ in detail to address new problems or concerns. See http://www.gnu.org/copyleft/.
Each version of the License is given a distinguishing version number. If the Document speci-
ﬁes that a particular numbered version of this License "or any later version" applies to it, you
have the option of following the terms and conditions either of that speciﬁed version or of any
later version that has been published (not as a draft) by the Free Software Foundation. If the
Document does not specify a version number of this License, you may choose any version ever
published (not as a draft) by the Free Software Foundation.
Ơ Please consider the environment before printing
Sciences.ch 13/3049

Draft
1.3 Only in non-free release
The free version of this PDF has only the 5,000 first pages. By purchasing the complete version
with 2,000 more pages you will get the complementary following subjects (always with the
same level of mathematical rigor and of details):
• Probabilites:
– Baysian conjugation for Normal and Binomial law
• Statistics:
– Mode and Median of statistical laws
– Semi-variance
– Partial and semi-partial correlation
– M-Estimators for localization and for dispersion
– Likelihood of censored data
– Jensen Inequality
– Normal Law Entropy
– Maximum likelihood Test
– Propension score
– Equivalence test
– Quasi-correlation matrix
– Factorial Analysis
– Hotelling T-Test
– Welch Test with Welch-Satterhwaite equation
– ANCOVA
– Wald-Wolfowitz Test (binary sequence)
– Odds Ratio and its confidence interval
– Risk Ratio and its confidence interval
– Fieller test for the ration of two means
– Ellipse of control
– Poisson Model for the average (2D) spatial distance
– Canonical Correlation
– Intraclass correlation coefficient (ICC)
– G-test of periodicity
– Gaussian and Student copula
– Hierarchical Fixed Factor ANOVA
– Square Latin ANOVA without replication
14/3049 Sciences.ch

Draft
– Introduction to MANOVA
– Extreme Values Theorem
– Survey Theory
• Sequences and Series:
– Properties of Fourier transforms
– Laplace Transform
– Z transform (common Z transforms, inverse common Z transforms)
• Differential Calculus
– O.D.E. classiﬁcation
– Lebesgue Integral with numerical application in MATLAB™
– Laplace Method
– Continuous and Discrete Convolution
• Functional Analysis:
– Convexity and Concavity of a function
• Complex Analysis:
– Residue Theorem for polynomial ratios
• Topology:
– Mahalanobis Distance
• Analytical Geoemtry:
– Classiﬁcation of ellipses with the determinant
• Differential Geometry:
– Normal coordinates
– Gauss curvature
– Isoperimetric plane theorem
• Mechanics:
– Inverse pendulum
– Magnus effect
• Optical Wave:
– Magnifying glass
– Fresnel Diffraction
– Fraunhoffer Diffraction
Sciences.ch 15/3049

Draft
• Astronomy:
– Kepler equation and time traveling
– MacCullagh’s formula
– Body flatness indirect calculation
• General Relativity:
– Real volume of an object in General Relativity
– Gravitational Waves
• Cosmology:
– Friedmann-Lemaitre metric
• Numerical Methods:
– Univariate optimization problem with substitution method
– Lagrange multiplier
– Acceptation/Rejection Sampling
– Gibbs Sampling
– Durbin-Watons test
– Linear Regression ANOVA (Regression Fisher Test)
– Multicolinearity analysis with Variance Inflation Factor (VIF)
– Outliers vs Influential values
– Generalized Linear Models (Gauss, Poissson, Negative Binomial, Gamma)
– Logistic regression based on maximum likelihood
– Cronbach coherence indicator
– Linear discriminant Analysis
– Quadratic discriminant Analysis
– Multidimensional scaling (MDS)
– Linear Mixture Model (LMM)
– Kernel Smoothing
– Mean Shift
– PLS Regression
– Factorial Analysis
– Correspondence Factorial Analysis
– GRG Generalized Reduced Gradient (GRG) optimization method
– Deming regression (orthogonal regression)
• Industrial Engineering:
– Operating characteristic curve for X-bar & p CC
16/3049 Sciences.ch

Draft
– Design of reliability tests (time tests, size tests)
– Average failure rate (AFR)
– Laney control chart
– Box-Behnken Design
– Square Latin Design (three level factors)
– Box Domains
– Central Composite Design
– Center Face Cube Design
– D-optimal/A-optimal and G-optimal designs
– Mixture Designs
– Weibull distribution linearization
– Cox Survival Model (Cox Proportional Hazard Model)
– Modelization by Structural Equations
– Accelerated life testing
– Markov Reliability Chain
• Electronics: Microelectronics
• Finance:
– Continuous Yield rate
– Zero-Coupon curve rates
– Equivalence of an obligation rate for a treasure bond
– Spot rate and Forward rate
– Adjusting the beta of a portfolio with Futures
– Cox-Ingersoll Future/Forward price equality
– Solution of Black & Scholes ODE
– Black Model
– Macaulay Duration
– Modified Duration
– Modified Internal Rate of Return (MIRR)
– Binomial Tree (Cox-Ross-Rubinstein)
– Options Portfolio hedging
* Protective Put/Call
* Bull Spread/Call
* Bear Spread/Call
* Butterfly
* Straddle
* Strangle
Sciences.ch 17/3049

Draft
* Collar
* Calendar spreads
* Portfolio allocation methods
· Optimal weighted portfolio for balanced risk
· Optimal weighted portfolio for error tracking
· Optimal weighted Sharp’s portfolio
· Optimal weighted portfolio with maximum diversiﬁcation
· Optimal market-bench weighted Treynor-Black Portfolio
– Surplus at Risk (SVaR)
– Default Credit Risk (based on Standard & Poor rating)
– VaR Equity Coverage
– Condition VaR loss (CVaR)
– Fokker-Planck equation
– ARCH-GARCH stochastic process
• Quantitative Management:
– Newvendor problem
– BullwhipEffect
– Condorcet paradox
– Computerized Relative Allocation of Facilities Technique (CRAFT)
– Real options
– Procedural Hierarchical Analysis
– Differed Capital in living case (life assurance)
– Modiﬁed Duration
– Death differed temporary (life assurance)
Remember that the full version of this book with perpetual free updates is available for the price
of $ 299.- each and for $ 499.- you get the LATEX sources (for information on purchase you can
simply send me an email).
As every robust product has a lifecycle. The lifecycle begins when a product is released and ends
when it’s no longer supported. Knowing key dates in this lifecycle helps you make informed
decisions about when to upgrade. This book has the following lifecycle: a new major or minor
version is published at each end of month (last day of month) following the Gregorian Calendar
and can be downloaded with the following link:
18/3049 Sciences.ch

Draft
The writing of this book took almost 15 years of time and effort (with average of 2 hours by
day), so please don’t forget to donate if you found this book useful or if you want I give for free
to everybody the complete version.
There are also other ways of supporting this book to ensure it is maintained and well supported
in the future! Linking/spreading the word, and submitting proofs contributions will all help.
Sciences.ch 19/3049

Draft
2Acknowledgements
The ideas in this book have been developed and reinforced by many people. I have greatly
benefited from my regular interactions with hundreds of executives from all backgrounds, in-
cluding CEOs, CFOs, PMs of many companies around the world, teaching essions, developing
company-specific programs, consulting, and even informal conversations. I am grateful to them
for sharing their wisdom with me and inspiring many of the ideas in the book
This book and its companion website would not have been possible without the valuable support
of the people mentioned below. They find here the expression of my gratitude (and for sure if
some errors remains in this book this is obviously their fault...):
• Guglielmetti Rafael, passionate about web development and old mathematics student at
EPFL (École Polytechnique Fédérale de Lausanne) in Switzerland, for the development
and improvement of the forum PHP code.
• Harmel Léon (†2012), graduate electrical engineer with a specialization in electronics
and automation, responsible in the physical research laboratory at ACEC in Charleroi, for
the provision of documentation that was used in the chapters of quantum particle, wave,
physics, quantum field and spinor calculus and general relativity.
• Legrand Mathias, Ph.D. École Centrale de Nantes for his help on the redaction of the
first 550 pages of the LATEX eBook version of the website.
• Ricchiuto Ruben, engineer degree in Physics HES (B.Sc.) from the Engineering School
of Geneva (CH) and mathematician from the University of Geneva for his valuable help in
plasma physics, electromagnetism, quantum physics, statistics, topology, quantum chem-
istry, fractals theory, analysis and many other areas affecting pure mathematics and com-
puting.
• Regulars participants to Les Mathematiques.net and Futura-Sciences.com forum, for their
valuable assistance in many areas of mathematics and physics. The debates and discus-
sions that took place on the forums helps to constantly improve the educational aspect of
this book.
• The Wikipedia and PlanetMath websites to whom I am indebted to many borrow almost
20

Draft
Vincent ISOZ [EAME v3.0-2013] 2. Acknowledgements
word by word (and this is mutual...).
And thanks to all readers, webmasters and teachers for their websites and quality documents
available for free and anonymously on the Internet and regular forum stakeholders. I sometimes
verbatim recovered their explanations that do not require additions or corrections. It’s proba-
bly needless to say that you should not assume that these people are in total agreement with
the scientific purposes views expressed in this book; and are not responsible for any errors or
obscurities that you might accidentally find in it.
Finally, I would like to thank a few colleagues and customers who were willing to give me their
comments to improve the content of this book. However, it is certain that it can still be improved
on many points.
A wink at Lala Valerie Geidarova. Your eyes are everyday my guiding star since we met. I’m
happy the Universe allowed your soul to stop by. I tried do to the Electrodynamics section the
best I could for you.
For any public feedback or comment you can use the guestbook associated to this PDF (for
questions please use the forum!):
httpXGGwwwFs™ien™esF™hGhtmlenGguestõokFphp
or if you want to do a private feedback or comment you can contact me by email.
Sciences.ch 21/3049

Draft
3Introduction
Contents
3.1 Forewords . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.2 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.2.1 Descartes’ Method . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.2.2 Archimedean Oath . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.3 Vocabulary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.3.1 On Sciences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.3.2 Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3.4 Science and Faith . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
22

Draft
Vincent ISOZ [EAME v3.0-2013] 3. Introduction
This book who first Edition has been published in 2001 is designed so that
the knowledge required to read it is as basic as possible. It is not necessary
to have a Ph.D. to consult it, you just have to know reasoning, to think
critically, to observe and have time...
"Simplicity is the seal of truth and it radiates beauty"
Albert EINSTEIN
3.1 Forewords
No human endeavor has had more impact than the science on our lives and our conception of
the world and ourselves. His theories, his conquests and results are all around us.
Omnipresent in industry (aerospace, imaging, cryptography, transportation, chemistry, algorith-
mic, etc.) or in the services (banking, insurance, human resources, projects, logistics, architec-
ture, communications, etc.), Applied Mathematics also appears in many other areas: surveys,
risk modeling, data protection, etc. The Applied Mathematics influence our lives (telecommu-
nications, transport, medicine, meteorology, music, project management) and contribute to the
resolution of current issues: energy, health, environment, climate, optimization, sustainable de-
velopment, etc. much more than any soft skill techniques or methodology! They great success
are their fabulous dispersion in the real world and their increasing integration in all human activ-
ities. We are going therefore to a situation where mathematics will no longer have the monopoly
of mathematics, but where economists, managers and commercials will all do mathematics.
As a former student in the field of engineering, I have often regretted the absence of a single
book fairly comprehensive, detailed (without going to the extreme...) and educational if possible
free (!) and portable (being personally a fan of eBooks...) containing at least a non exhaustive
idea of the overall program of Applied Mathematics in engineering schools with an overview
of what is used for real in companies with more intuitive than rigorous proofs but with enough
details to avoid unnecessary effort to the reader. Also a book that does not require to adapt each
time a new notation or terminology specific to the author when it is not outright to change to
a a foreign language... and where anyone can suggest improvements or additions (through the
forum, guestbooks or by e-mail).
I was also frustrated during my studies to have quite often have to swallow "formulas" or "laws"
supposedly (and wrongly) unprovable or too complicated as my teachers says or even disap-
pointed by renowned authors books (where developments which are left to the reader or as
exercise and no real applications are even mention...). In this book predominates the will to
never confuse the reader with empty sentences like "it is evident that...", "it is easy to prove
that...", "we leave it to the reader as an exercise...", since all developments are presented in
detail. But I’m not a purist of maths! I have only one ambition: to explain the easiest way
possible.
Although I have to admit that prove some mathematical relations presented within the engi-
neering schools curriculum can not be done because of a lack of time in the official program
or size limit in a book, I can not accept that a teacher or author tells his student (respectively,
the reader) that certain laws are unprovable (because most of the time this is not true) or that
Sciences.ch 23/3049

Draft
3. Introduction Vincent ISOZ [EAME v3.0-2013]
such or such proof is too complicated without giving a reference (where the student can find the
information necessary to satisfy his curiosity) or at least a simplified but satisfactory proof.
Moreover, I think that it is totally archaic today that some teachers continue to ask to their
students to take a massive amount of notes. It would be much more favorable and optimal to
distribute a course handout containing all the details in order to be able to concentrate on the es-
sentials points with students, that is to say the oral explanations, interpretations, understanding,
reasoning and practice rather than excessive blackboard copy... Obviously by giving a complete
course handout some students will be brilliant by their absence but ... it is the better! Thus,
those who are passionate can deepen subjects at home or at the university library, the weak do
what they have to do and the rest (struggling students but workers) will follow the course given
by the teacher to profit to ask questions rather than mindlessly copying a blackboard.
Inspired on a learning model of an American scholar, whose I forgot the name (...) this book
proposes and imposes the following properties to the reader: discover, memorize, cite, integrate,
explain, restate, infer, select, use, decompose, compare, interpret, judge, argue, model, develop,
create, search, reasoning, develop in a clear progressive teaching way to develop the analytic
skills and openness.
So, in my mind, this non-exhaustive book (and its associated companion PDFs) must be a sub-
stitute, free of charge for all students and employees around the World, to many references
and gaps of the system, allowing any curious student not to be frustrated for many years dur-
ing his school education. Otherwise, the science of the engineer could have the aspect of a
frozen science, apart from the scientific and technical developments, a heteroclit accumulation
of knowledge and especially of formulas which made he considered as a tasteless subproduct
of mathematics and that brings companies to many false results...
This book has also been designed to meet the needs of executives, both finance as well as
non-finance managers. Any executive who wants to probe further and grasp the fundamentals
of strategic finance, strategic marketing or project management engineering and supply chain
issues will benefit from this book.
Obviously Applied Mathematics is such an abundant topic that a book of this scale can only
accommodate the basis. Readers are certainly encourage to go beyond this.
Those who see Applied Mathematics only as a tool (what it also is), or as the enemy of religious
beliefs, or as a boring school field school, are legion. However, it is perhaps useful to recall that,
as Galileo said, "the book of nature is written in the language of mathematics" (without wishing
to do scientism!). It is in this spirit that this book discusses Applied Mathematics for students in
the Natural, Earth and Life sciences, as well as for all those who have an occupation related to
the various subjects including philosophy or for anyone curious to learn about the involvement
of science in everyday life.
The choice to study engineering in this book as a branch of Applied Mathematics comes from
the fact that the differences between all areas of physics (formerly known as "natural philos-
ophy") and mathematics are so hardly notable that Fields medal (the highest award today in
the field of mathematics) was awarded in 1990 to physicist Edward Witten, who used physical
ideas to prove a mathematical theorem. This trend is certainly not fortuitous, because we can
observe that all science, since it seeks to achieve a more detailed understanding of the subject it
24/3049 Sciences.ch

Draft
studies, is always finish his trials in the pure mathematics (the absolute path by excellence ...).
Thus, we can predict in a far future, the convergence of all the sciences (pure, exact or social) to
the mathematics for the modelisation techniques (see for example the French PDF "L’explosion
des mathématiques" available in the download page of the companion website).
It can sometimes seem to us difficult (due to irrational as obscure and unjustified fear of pure
sciences in a large fraction of our contemporaries) to transmit the feeling of the mathematical
beauty of nature, its deepest harmony and the well-oiled mechanics of the Universe, to those
who know only the basics of algebra. The physicist R. Feynman spoke a day of "two cul-
tures": people who have and those who do not have sufficient understanding of mathematics
to appreciate the scientific structure of nature. It is a pity that mathematics are necessary to
deeply understand nature and that they also have a bad reputation. For the record, it is claimed
that a King who asked Euclid to teach him geometry complained about its difficulty. Euclid
replied, "There is no royal road". Physicists and mathematicians can not convert themselves to
a different language. If you want to learn about nature, to appreciate its true value, you must
understand its language. The nature is revealed only in this form and we can not be pretentious
to the point of asking him to change this fact.
In the same way, no intellectual discussion will allow you to communicate with a deaf per-
son what you really feel while listening music. Similarly, all discussion of the world remain
powerless to transmit an intimate understanding of the nature of those of the "other culture".
Philosophers and theologians may try to give you qualitative ideas about the Universe. The
fact that the scientific method (in the full sense of the term) can not convince the world of its
truth and purity, is perhaps the fact of the limited horizon of some people who imagine that
the human or another intuitive concept, sentimental or arbitrarily is the center of the Universe
(anthropocentric principle).
Of course, in order to share this mathematical knowledge, it is paradoxical to increase, with
our work, the long list of books already available in libraries, in commerce and on the Internet.
Nevertheless, we must be able to present arguments that justifies the creation of such a book
(and its associated website) as compared to books such as Feynman, Landau or Bourbaki and
Wikipedia/Wolfram themselves or Khan Academy. So what do we think I can add to such a
wealth of material?
1. The great pleasure that we take to this work ("keep the hand" and improve our skills) and
have a detailed compendium of tools for our customers.
2. The passion for sharing knowledge for free (battle again "copyright madness" and without
frontiers with a tool of quality as LATEX (at the opposite of Wikipedia that mixes LATEX and
normal text and the awful content of Khan Academy).
3. We want to offer Applied Mathematics in an enjoyable and easy-to-learn manner, because
we believe that Applied Mathematics change the way we understand the Universe.
4. This book was written in English before the French version of Wikipedia had good math-
ematical content (in 2001) and long before Khan Academy.
5. The quick updates/corrections opportunities (at the opposite of Khan Academy) and col-
laborations of a free e-book (with associated effective search tools) without having topics
that disappears (at the opposite of Wikipedia).
Sciences.ch 25/3049

Draft
6. The content depending on readers requests/comments and on our interests (at the opposite
of Khan Academy)!
7. At the opposite of Scientific publications (PRL or other similar) that sucks because don’t
give detailed proofs and sometimes turn in an infinite loop in references.
8. The access to LATEX sources to everybody so nobody need to recreate the wheel and loose
hundred or thousand of hours on redaction instead of innovation!
9. Rigorous presentation with simplified detailed proofs of all presented concepts (at the
opposite of Wikipedia and Khan Academy).
10. The presentation of many advanced and detailed mathematical tools used in business in
R&D.
11. The opportunity for students and teachers to reuse content by copy/paste (at the opposite
of Khan Academy).
12. Constant and fixed notation (at the opposite of Wikipedia and Khan Academy) throughout
the book, for mathematical operators, a clear language on all topics (3.C. criterion: clear,
complete and concise)
13. Gather as much information about pure and exact sciences in one electronic (portable),
homogeneous and rigorous book.
14. Release from all pseudo-truths, only truths that can be proven.
15. Benefit from the development of teaching methods that use the Internet to search for the
solution of mathematical problems.
16. The dramatic improvement of automatic translation software and computing power that
will make of this book, at least we hope, a reference in the fields of sciences.
17. and... because Applied Mathematics are beautiful and especially when written in LATEX
And also ... I believe that the results of individual research are the property of humanity and
should be available to all those who explore anywhere the phenomena of nature. In this way the
work of each benefit to all, and that is for all humanity that our knowledge cumulates and this
is the trend that allows Internet.
I do not hide that my contribution is limited largely to this day to that of a collector who gleans
his information in the works of masters or publications or from anonymous web pages and
who completes and argues developments and improved them when this is possible. For those
who would accuse me of plagiarism, they should think on the fact that the theorems presented
in most non-free books and commercially available have been discovered and written by their
predecessors and their own personal contribution was also made, like mine, to put all this in-
formation in a clear and modern form a few hundred years later. In addition, it can be seen as
doubtful that we ask to pay for access to a culture that is certainly the only truly valid and fair
one in this world and where there is no patent or intellectual property rights.
This book also reflects my own intellectual limitations. Although I try to study as much science
and math fields as possible, it is impossible to master them all. This book shows clearly only
26/3049 Sciences.ch

Draft
my own interests and experiences as consultant, but also my strengths and my weaknesses. I
am responsible for the selection of inputs and, of course, of possible errors and imperfections.
After attempting a strict (linear) order of presentation of the subject, I decided to arrange this
book in a more pedagogical (thematic) way and always with practical examples o applications.
It is in my opinion very difficult to speak of so vast subject in a purely mathematical order in
only one human life, that is to say, when the concepts are introduced one by one, from those
already known (where each theory, operator, tools, etc.. would not appear before its definition).
Such a plan would require cutting the book, in pieces that are not more thematic. So I decided
to present things in a logical order and not in order of need. Thus the reader will encounter, as
the editor himself, to the extreme complexity of the subject.
The consequences of this choice are the following:
1. Sometimes it will necessary to admit certain concepts, even to understand later.
2. It will probably be necessary for the reader to go at least twice throughout the book. At
the first reading, we apprehend the essential and at the second reading, we understand the
details (I congratulate this who understand all the subtleties the first time).
3. You must accept the fact that some topics are repeated and that there are many cross-
references and complementary remarks.
Some know that for every theorem and mathematical model, there are almost always several
methods of proofs. I’ve always tried to choose the one that seemed the most simple (e.g. in
relativity and quantum physics there is the algebraic and matrix formalism). The objective is to
arrive at the same result anyway.
This book being in its draft version, it necessarily has lacks on convergence controls, on continu-
ity, grammar and others... (which will horrify some readers and mathematicians ...)! However,
I have avoided (or, otherwise, I indicate it) the usual approximations of physics and the use
of dimensional analysis, by using it as little as possible. I also try to avoid as much as possi-
ble subjects with mathematical tools that have not previously been presented and demonstrated
rigorously.
Finally, this presentation, that can still be improved, is not an absolute reference and contains
errors. Any comment is welcome. I shall endeavour, as far as possible, to correct the weaknesses
and make the necessary changes as soon as possible.
However, while mathematics is accurate and indisputable, theoretical physics (its models), is
still interpreted in the common vocabulary (but not in the mathematical vocabulary) and its
conclusions all relative. I can only advise, when you read this book, to read by for yourself
and not to be subjected to outside influences. You must have a very (very) critical mind, take
nothing for granted and question everything without hesitation. In addition, the keyword of
good scientist should be: "Doubt, doubt, doubt ... doubt still, and always checks.". We also
recall that "nothing that we can see, hear, smell, touch or taste, is what it seems to be", therefore
do not rely on your daily experience to draw hasty conclusions, be critical, Cartesian, rational
and rigorous in your development, reasoning and conclusions!
Sciences.ch 27/3049

Draft
I want to say to those who would try to ﬁnd themselves the results of some developments of this
book, do not worry if they do not success or if they doubt about their competences because of
the time spent solving an equation or problem: some theories that seem obvious or easy today,
have sometimes needed several weeks, months, even years, to be developed by mathematicians
or leading physicists in the past!
I also tried to ensure that this book is pleasing to the eye and to read through.
Finally, I have chosen to write this work in the ﬁrst person plural form ("we"). Indeed, the
mathematical physics is not a science that has been made or has evolve through individual work
but with intensive collaboration between people connected by the same passion and desire of
knowledge. Thus, by making use of "we", I would like pay tribute to the dead and missing
scientists, to contemporary and future researchers for the work they will perform in order to
approach the truth and wisdom.
28/3049 Sciences.ch

Draft
3.2 Methods
Science is the set of all systematic efforts (scrupulous observations and plausible assumptions
until the evidence of the contrary) to acquire knowledge about our environment, to organize and
synthesize them into testable laws and theories, whose main purpose is to explain the "how" of
things (and NOT the why!) often by a three-step approach:
− What do we have?
− Where will we go?
− What is our goal?
Scientists have to submit their ideas and results to independent verification and replication of
their peers. They must abandon or modify their conclusions when confronted with more com-
plete or different evidences. The credibility of Science is based on this self-correcting mecha-
nism. The history of science shows that this system works very long and very well compared
to all the others. In each area, progress has been spectacular. However, the system sometimes
failed and has also to be corrected before small drifts accumulate.
The downside is that scientists are humans. They have the imperfections of all humans, and
especially, vanity, pride and conceit. Nowadays, it happens that many people working on the
same topic for a given time develop a common faith and believe they hold the truth. The leader
of the faith is the Pope and distills his opinion. The Pope that plays the game, takes his miter
and his pilgrim’s staff to evangelize his fellow heretics. Until then, this makes smile. But, as
in real religions, they sometimes annoying to want to expand their opinion to those who do not
believe. Some of these "churches" do not hesitate to behave like the Inquisition. Those who dare
to express a different opinion are burned at every opportunity, during conferences, or at their
place of work. Some young researchers, uninspired, prefer to convert to the dominant religion,
to become clerics faster rather than innovative researchers or even iconoclasts. The great Pope
write his Bible to disseminate his ideas, imposes it to read to students and newcomers. He
formats then the thought of younger generations and ensures his throne. This is a medieval
attitude that can block progress. Some Popes go so far that they believe be the pope in their
specialization field automatically gives them the same throne in all other areas...
This warning, and the reminders that will follow, must serve the scientific to ask himself by
making good use of what we consider today as the good working practices (we will discuss
the principles of the Descartes method more below) to solve problems or develop theoretical
models.
Sciences.ch 29/3049

Draft
For this purpose, here is a summary table that provides the steps that should be followed by a
scientific who works in mathematics or theoretical physics (for definitions, see just below):
Mathematics Physics
1. Expose formally or in common language
the "hypothesis", the "conjecture" the "prop-
erty" to prove (hypothesis are noted H1.,
H2., etc. the conjectures CJ1., CJ2., etc. and
the properties P1., P2., etc.).
1. Expose correctly in a formally or common
language all the details of the "problems" to
solve (problems are noted P1., P2., etc.).
2. Define the "axioms" (non-demonstrable,
independent and non-contradictory) that
will give the starting points and establish
restrictions on development (the axioms are
noted A1., A2, etc.)1
.
In the same vein, the mathematicians
defines the specialized vocabulary related
to mathematical operators which will be
denoted by D1., D2., etc.
2. Define (or state) the "postulates" or "prin-
ciples" or the "hypothesis" and "assump-
tions" (supposedly unprovable...) that will
give the starting point and establish restric-
tions on the developments (typically, as-
sumptions and principles are denoted P1.,
P2., etc. and assumptions H1., H2., etc. try-
ing to avoid the notation confusion between
postulates and principles)2
.
3. Once the Axioms laid, pull directly "lem-
mas" or "properties" whose validity follows
directly and prepare the development of the-
orem supposed to validate departure hypoth-
esis or conjectures (Lemmas being denoted
L1., L2., etc. and properties P1., P2., etc.).
3. Once the "theoretical model" devel-
oped, check equations units for possible er-
rors in the developments (such checks being
marked VA1., VA2., etc.).
4. Once the "theorems" (noted T1., T2.,
etc.) prooved conclude on "consequences"
(denoted C1., C2., etc.) and even properties
(noted P1., P2., etc.).
4. Search for borderline cases (including
"singularities") of the model to verify the
validity intuitively (these borderline controls
are noted CL1., CL2., etc.).
5. Test the strength (robustness) or use-
fulness of the conjectures or hypothesis by
proving the reciprocal of the theorem or
by comparing them with other examples of
mathematical well-know theories to see if
form together a coherent structure (examples
being denoted E1., E2., etc.).
5. Experimentally test the theoretical model
obtained and submit work to compare with
other independent research teams. The new
model should provide experimental results
and never observed (predictions to falsify).
If the model is validated then it is the official
status of "theory".
6. Possible remarks may be shown in a hi-
erarchically structured order and noted R1.,
R2., etc.
6. Possible remarks may be shown in a hi-
erarchically structured order and noted R1.,
R2., etc.
Table 3.1 – Methodology for Maths & Physics Developments
Proceed as in the above table is a possible working basis for people working in mathematics
and physics. Obviously, proceed cleanly and traditionally as above takes a little more time than
1
Sometimes "properties", "conditions" and "axioms" are confused while the concept of axiom is much more accu-
rate and profound.
2
You should not forget, however, that the validity of a model is not dependent on the realism of its assumptions but
on the conformity of its implications with reality.
30/3049 Sciences.ch

Draft
doing things no matter how (this is why most teachers do not follow these rules, they don’t have
enough time to cover the entire course program).
Note also a fun shape of scientific 8 commandments:
1. The phenomenas you will observe
And never measures you will falsify
(attention to the confirmation error: study only phenomena that validate your belief)
2. Hypothesis you will proposed
That with experiment you will test
3. The experiment precisely you will describe
Because your colleague will reproduce it
(attention to the narrative discipline trap: the facts will be fitted to the desired results)
4. With your results
A theory you will build
5. Parsimony you will use
And the simplest hypothesis you will retain
6. Ultimate truth will never be (epistemic humility)
And always you will search for the truth
Sciences.ch 31/3049

Draft
7. From a non-refutable thesis you will refrain
Because outside of the science it will remain
8. All failures will be like a success
Because science can confirm but also invalidate
Remarks
R1. Caution! It is very easy to make new physical theories by just aligning words. This
is named "philosophy" and the Greeks thought of the atoms in this method. This can lead
with a lot of luck to a true theory. Against it is much more difficult to make a "predictive
theory", that is to say with equations that predict the outcome of an experiment.
R2. What separates mathematics and physics is that in mathematics, the hypothesis is
always true. Mathematical discourse is not a proof of an external seeking truth, but a
target of consistency. What should be correct is just the reasoning.
When these rules are not respected, we speak of "scientific fraud" (which often leads to being
fired from his job but unfortunately we still not retired the diplomas when it happens). In
general, scientific fraud itself comes in three main forms: plagiarism, fabrication of data and
alteration of results unfavourable to the hypothesis, the omission of clear working hypotheses
and recolted datas. To these frauds we can also add behaviors that pose problems regarding to
the quality of work or more specifically to ethics, such as those aimed at increasing appearance
in the production (and through the famous of the scientist) by submitting for example several
times the same publication with only a few modifications, the omission of conflict of interest,
the dangerous experiments, the non-conservation of primary data, etc.
3.2.1 Descartes’ Method
Now we present the four principles of the Descartes’ method which, as remind, is considered
as the first scientific in history by his method of analysis:
P1. Never accept anything as true that I obviously knew her to be such. That is to say, care-
fully avoid precipitation and to understand nothing more in my judgements than what
would appear so clearly and distinctly to my mind, that I had no occasion to doubt.
P2. Divide each of the difficulties I have to examine into as many parts as possible (scrupulous
observations and plausible hypothesis until evidence of the opposite), and that would be
required to resolve them in the best way.
P3. Driving my thoughts in order, beginning with the simplest objects and easiest to know, to
go up gradually by degrees to the knowledge of the most compounds, and even assuming
the order between those who not naturally precede each other.
P4. Make everywhere so complete enumerations and so genera reviews, that I’m sure not to
omit anything.
32/3049 Sciences.ch

Draft
3.2.2 Archimedean Oath
Inspired by the Hippocratic Oath, a group of students of the Ecole Polytechnique Fédérale de
Lausanne in 1990 developed an oath of Archimedes expressing the responsibilities and duties
of the engineer and technician. It was taken in various versions by other European engineering
schools and could serve as basic inspiration oath for scientiﬁc researchers (even if there are
some important points).
"Considering the life of Archimedes of Syracuse which illustrated as of Antiquity the ambiva-
lent potential of the technique, considering the responsibility increasing for the engineers and
scientists with regard to the men and nature, considering the importance of the ethical problems
that the technique and its applications raise, today, I pledge following and will endeavour to
tend towards the ideal which they represent:
1. I will practise my profession for the good of the people, in the respect of the Rights of the
Man and of the environment.
2. I will recognize, being as well as possible informed to me, the responsibility for my acts
and will not discharge me to in no case on others.
3. I will endeavour to perfect my professional competences.
4. In the choice and the realization of my projects, I will remain attentive with their context
and their consequences, in particular from the point of view technical, economic, social,
ecological... I will pay a detailed attention to the projects being able to have ﬁne soldiers.
5. I will contribute, in the measurement of my means, to promote equitable relationship
between the men and to support the development of the countries lower-income group.
6. I will transmit, with rigor and honesty, with interlocutors chosen with understanding,
any information important, if it represents an asset for the company or if its retention
constitutes a danger to others. In the latter case, I will take care that information leads to
concrete provisions.
7. I will not let myself dominate by the defence of my interests or those of my profession.
8. I will make an effort, in the measurement of my means, to lead my company to take into
account the concerns of this Oath.
9. I will practise my profession in all intellectual honesty, with conscience and dignity.
10. I promise it solemnly, freely and on my honour."
Sciences.ch 33/3049

Draft
3.3 Vocabulary
Physics and mathematics, like any field of specialization, has its own vocabulary. So that the
reader is not lost in the understanding of certain texts he can read in this PDF, we chosed to
present here a few fundamentals words, abbreviations and definitions to know.
Thus, the mathematician like to finish his proofs (when he thinks they are correct) by the abbre-
viation "Q.E.D." which means "Quod Erat demonstrandum" (this is Latin).
And during definitions (they are many in math and physics ...) scientist often use the following
terminology:
• ... it is sufficient that ...
• ... if and only if ...
• ... necessary and sufficient ...
• ... means ...
The four are not equivalent (identical in the strict sense). Because "it is sufficient that" corre-
spond to a sufficient condition, but not to a necessary condition.
3.3.1 On Sciences
It is important that we define rigorously the different types of sciences which human been often
refers. Indeed, it seems that in the 21st century a misnomer is established and that it became
impossible for people to distinguish the "intrinsic quality" between a science and another one.
Remark
Etymologically, the word "science" comes from the Latin "Scienta" (knowledge) whose
root is the verb "scire" which means "to know".
This abuse of language is probably the fact that pure and accurate sciences lose their illusions
of universality and objectivity, in the sense that they are self-correcting. This has for effect that
some sciences are relegated to the background and try to borrow these methods, principles and
origins to create confusion. We must therefore be very careful about the claims of scientificity
in the human sciences, and this is also (or especially) true for the dominant trends in economics,
sociology and psychology. Quite simply, the issues addressed by the human sciences are ex-
tremely complex, poorly reproducible, and empirical arguments supporting their theories are
often quite low.
By itself, however, science does not produce absolute truth. By principle, a scientific theory
is valid as long as it can predict measurable and reproducible results. But the problems of
interpretation of these results are part of natural philosophy.
34/3049 Sciences.ch

Draft
Given the diversity of phenomena to be studied, over the centuries there has been a growing
number of disciplines such as chemistry, biology, thermodynamics, etc. All these disciplines
that are a priori heterogeneous have common foundation physics, for language mathematics and
for elementary principle the scientific method.
Thus, a small memory refresh seems useful:
Definitions:
D1. We define as "pure science" any set of knowledge based on rigorous reasoning valid what-
ever the (arbitrary) elementary factor selected (when we say then "independent of sensi-
ble reality") and restricted to the minimum necessary. Only mathematics (often called the
"queen of sciences") that can be classified in this category.
D2. We define as "exact science" or "hard science", any set of knowledge based on the study
of an observation, observation that has been transcribed in symbolic form and that can
be reproduce (theoretical physics for example... sometimes...). Primarily, the purpose of
exact sciences is not to explain the "why" but the "how".
And never forget... Science (especially physics) doesn’t have to "make sense" it just has
to make all the right, testable predictions!
According to the philosopher Karl Popper, a theory is scientifically acceptable if, as presented,
it can be falsifiable, i.e. subjected to experimental tests. The "scientific knowledge" is then by
definition the set of theories that have resisted to falsification. Science is by nature subject to
continuous questioning.
Caution! There is no doubt that the exact sciences have yet an enormous prestige, even among
their opponents because of their theoretical and practical success. It is certain that some scien-
tists sometimes abuse of this prestige by showing a sense of superiority that is not necessarily
justified. Moreover, it often happens that this same scientists exposed in the popular literature,
very speculative ideas as if they were very approved, and extrapolate their results outside the
context in which they were tested (and ... under the hypotheses they were checked once...).
Remark
The two previous definitions are often included in the definition of "deductive sciences"
or even "phenomenological science".
D3. We define as "engineering science" any set of knowledge or practices applied to the needs
of human society such as electronics, chemistry, computer science, telecommunications,
robotics, aerospace, biotechnology...
D4. We define as "science" any body of knowledge based on studies or observations of events
whose interpretation has not yet been transcribed and verified with mathematical rigour,
characteristic of previous sciences, but using comparative statistics. We include in this
definition: medicine (we should however be careful because some parts of medicine are
Sciences.ch 35/3049

Draft
studying phenomena using mathematical descriptions such as neural networks or other
phenomena associated with known physical causes), sociology, psychology, history, biol-
ogy, etc.
D5. We define as "soft science" or "para-science", any set of knowledge or practices that are
currently based on non-verifiable facts (not scientifically reproducible) by experience or
by mathematics. We include in this definition: astrology, theology, paranormal (which
was demolished by zetetic science), graphology...
As some scientists say: «It looks like science, it use the vocabulary of science... but that’s
not science at all.»
D6. We define as "phenomenological science" or "natural sciences", any science which is not
included in the above definitions (history, sociology, psychology, zoology, biology, ...)
D7. "Scientism" is an ideology that considers experimental science is the only valid mode
of knowledge, or, at least, superior to all other forms of interpretation in the world. In
this perspective, there is no philosophical, religious or moral truths superior of scientific
theories. Only account what is scientifically proven.
D8. "Positivism" is a set of ideas that considers that only the analysis and understanding of
facts verified by experience can explain the phenomena of the sensible world. Certainty
is provided solely by the scientific experiment. He rejects introspection, intuition and
metaphysical approach to explain any knowledge of the phenomena.
What is interesting about this doctrine is that it is certainly one of the few that re-
quires people to have to think for themselves and to understand the environment around
them by continually questioning everything and by never accepting anything as granted
(...). In addition, the real sciences have this extraordinary property that they give the
possibility to understand things beyond what we can see.
But, science is science, and nothing more: a certain ordering, not too bad success, things that
no longer leads to the metaphysics as the time of Aristotle, but that does not pretend to give us
the whole story on reality or even the bottom of visible things.
36/3049 Sciences.ch

Draft
3.3.2 Terminology
The table of methods we presented above contains terms that may perhaps seem unknown or
barbarians for you. This is why it seems important to provide definitions of these and some
other equally important that can avoid important confusion.
Definitions:
D1. Beyond its negative sense, the idea of "problem" refers to the first step of the scientific
method. Formulate a problem is also essential for its resolution and allows to properly
understand what is the problem and see what needs to be resolved.
The concept of problem is intimately connected to the concept of "assumption"
that we will see the definition below.
D2. A "hypothesis" is always, in the context of a theory already established or underlying, a
supposition awaiting confirmation or refutation that attempts to explain a group of facts
or predict the onset of new facts.
Thus, a hypothesis can be at the origin of a theoretical problem that has to be
resolved formally.
D3. The "postulate" in physics corresponds frequently to a principle (see definition below)
which admission is required to establish a proof (we mean that this is a non-provable
proposition).
The mathematical equivalent (but in a more rigorous version) of the assumption is
the "axiom" for which we will see the definition below.
D4. A "principle" (close parent of "postulate") is a proposal accepted as a basis for reasoning
or a general theoretical guide line for reasoning that needs to be performed. In physics,
it is also a general law governing a set of phenomena and verified by the accuracy of its
consequences.
The word "principle" is used with abuse in small classes or engineering schools
by teachers not knowing (which is very rare), or unwilling (rather common), or that can’t
because lack of time (almost exclusively ) prove a relation.
The equivalent of the postulate or principle in mathematics is the "axiom" which
we define as follows:
D5. An "axiom" is a self-evident proposition or truth by itself which admission is necessary
to establish a proof.
Sciences.ch 37/3049

Draft
Remarks
R1. We could say that this is something we define as the truth for the speech that we
argue, like a rule of the game, and that it does not necessarily a universal truth value in
the sensitive world around us.
R2. Axioms must always be independent (one should not be able to be proved from the
other) and non-contradictory (sometimes we also say that they must be "consistent").
D6. The "corollary" is a term unfortunately almost nonexistent in physics (wrongly!) and that
is in fact a proposal resulting from a truth already demonstrated. We can also say that
a corollary is and obvious and necessary consequence of a theorem (or sometimes of a
postulate in physics).
D7. A "lemma" is a proposal deduce from one or more assumptions or axioms and that for
which the proof prepares this of a theorem.
Remark
The concept of "lemma" is also (and this is unfortunate) almost used only in the field of
mathematics.
D8. A "conjecture" is a supposition or opinion based on the likelihood of a mathematical
result.
Many conjectures have as as little similar to lemmas, as they are checkpoints to
obtain significant results.
D9. Beyond its weak conjecture sense, a "theory" or "theorem" is a set articulated around a
hypothesis and supported by a set of facts or developments that give it a positive content
and make the hypothesis well-founded (or at least plausible in the case of theoretical
physics).
D10. A "singularity" is an indeterminacy in a calculation That takes the appearance of a division
by zero. This term is both used in mathematics and in physics.
D11. A "proof" is a set of mathematical procedures to follow to prove the result already known
or not of a theorem.
D12. If the word "paradox" etymologically means: contrary to common opinion, it is not by
pure taste for provocation, but rather for solid reasons. A "sophism" meanwhile, is a
deliberately provocative statement, a false proposition based on an apparently valid rea-
soning. Thus we speak about the "Zeno’s paradox" when in reality it is only a sophism.
The paradox is not limited to falsity, but implies the coexistence of truth and falsity, so that
one can no longer distinguish true and the false. The paradox appears as an unsolvable
problem an "aporia".
38/3049 Sciences.ch

Draft
Remark
It should be added that the well-knows paradoxes, by the questions they raised, have per-
mitted signiﬁcant advances to science and led to major conceptual revolutions in math-
ematics as in theoretical physics (the paradoxes on sets and on inﬁnity in mathematical,
and those at the base of relativity and quantum physics).
Sciences.ch 39/3049

Draft
3.4 Science and Faith
We will see that in Science, a theory is usually incomplete because it can not fully describe the
complexity of the real world or because it does not predict what we don’t know (excepted for
Quantum Physics or General Relativity). It is thus for theories like the Big Bang (see chapter on
Astrophysics) or the Evolution of species (see sections on Populations Dynamics or Population
Theory or even Games Theory) because they are not reproducible in laboratories under identical
conditions. But some other theories are so accurate to predict physical phenomena that some
people believe that mathematics is the nearest language with God (at least for those that believe
in a divinity...).
We should distinguish between different scientiﬁc currents:
• "Realism" is a doctrine where physical theories have the aim to describe reality as it is in
itself, in its unobservable components.
• "Instrumentalism" is a doctrine where theories are only tools to predict observations but
do not describe reality itself.
• "Fictionalism" is the doctrine where the content repository (principles and postulates) of
theories is just an illusion, useful only to ensure the linguistic articulation of the funda-
mental equations.
40/3049 Sciences.ch

Draft
Even if today the scientific theories are sponsored by many specialists, alternative theories have
valid arguments and we can not totally dismiss them. However, the creation of the world in
seven days as described in the Bible is difficult to accept, and many believers recognize that a
literal reading of the Bible is not compatible with the current state of our knowledge and that
is more prudent to interpret it as a parable. If science never provides definitive answer, it is no
longer possible to ignore it.
Faith (whether religious, superstitious, pseudo-scientific or other) on the contrary is intended
to provide absolute truths of a different nature as it is a personal unverifiable belief. In fact,
one of the functions of religion is to give meaning to the phenomena that can not be explained
rationally. Progress of knowledge trough science therefore cause sometimes questioning the
religious dogma.
Conversely, except try to impose his own faith (which is nothing but a subjective and intimate
personal conviction ) to others, we must defy the natural temptation to characterize scientifically
proven fact extrapolations of scientific models beyond their scope.
The word "science" is, as we have already mentioned above, increasingly used to argue that
there is a scientific evidence where there is only a belief (some web pages like this proliferate
always more and more). According to its detractors it is, for example, the case of the movement
of Scientology (but there are many others). According to them, we should rather speak about
"occult sciences".
The occult sciences and traditional sciences exist since antiquity; they consist on a series of
mysterious knowledge and practices designed to penetrate and dominate the secrets of nature.
Over the past centuries, they have been progressively excluded from science. The philosopher
Karl Popper has longly questioned himself about the nature of the demarcation between science
and pseudoscience. After noticing that it is possible to find observations to confirm almost any
theory, he proposes a methodology based on falsifiability. A theory must according to him, to
deserve the adjective "scientific", guarantee the impossibility of some events. It becomes there-
fore rebuttable, so (and only then) capable of integrating science. It would suffice to observe
any of these events to invalidate the theory, and therefore take the way to improving it.
Finally, we would like to quote Lavoisier: «The physicist may also, in the silence of his labo-
ratory and his cabinet, perform patriotic functions; he can thanks to his works reduce the mass
of evils which afflict happiness and, had he not, contributed by the new roads that he opened to
himself, only to delay of a few years, of a few days, the average life of humans, he could also
aspire to the glorious title of benefactor of humanity.»
Section quality score: 151 votes, 75.23%
Sciences.ch 41/3049

Draft
4Arithmetic
Mathematics is the ultimate form of forced art. (unknown)
Contents
4.1 Proof Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
4.1.1 Paradoxes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.1.2 Propositional Calculus . . . . . . . . . . . . . . . . . . . . . . . . 52
4.1.3 Predicate Calculus . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
4.2 Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
4.2.1 Digital Bases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
4.2.2 Type of Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
4.3 Arithmetic Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
4.3.1 Binary Relations . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
4.3.2 Fundamental Arithmetic Laws . . . . . . . . . . . . . . . . . . . . 144
4.3.3 Arithmetic Polynomials . . . . . . . . . . . . . . . . . . . . . . . . 160
4.3.4 Absolute Value . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
4.3.5 Calculation Rules (operators priorities) . . . . . . . . . . . . . . . . 164
4.4 Number Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
4.4.1 Principle of good order . . . . . . . . . . . . . . . . . . . . . . . . 169
4.4.2 Induction Principle . . . . . . . . . . . . . . . . . . . . . . . . . . 170
4.4.3 Divisibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
4.5 Set Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
4.5.1 Zermelo-Fraenkel Axiomatic . . . . . . . . . . . . . . . . . . . . . 198
4.5.2 Set Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
4.5.3 Functions and Applications . . . . . . . . . . . . . . . . . . . . . . 217
4.6 Probabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233
4.6.1 Event Universe . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
4.6.2 Kolmogorov’s Axioms . . . . . . . . . . . . . . . . . . . . . . . . 235
42

Draft
Vincent ISOZ [EAME v3.0-2013] 4. Arithmetic
4.6.3 Conditional Probabilities . . . . . . . . . . . . . . . . . . . . . . . 241
4.6.4 Martingales . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260
4.6.5 Combinatorial Analysis . . . . . . . . . . . . . . . . . . . . . . . . 262
4.6.6 Markov Chains . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
4.7 Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276
4.7.1 Samples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278
4.7.2 Averages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
4.7.3 Type of variables . . . . . . . . . . . . . . . . . . . . . . . . . . . 301
4.7.4 Fundamental postulate of statistics . . . . . . . . . . . . . . . . . . 329
4.7.5 Diversity Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329
4.7.6 Distribution Functions (probabilities laws) . . . . . . . . . . . . . . 331
4.7.7 Likelihood Estimators . . . . . . . . . . . . . . . . . . . . . . . . . 433
4.7.8 Finite Population Correction Factor . . . . . . . . . . . . . . . . . . 445
4.7.9 Conﬁdence Intervals . . . . . . . . . . . . . . . . . . . . . . . . . 448
4.7.10 Weak Law of Large Numbers . . . . . . . . . . . . . . . . . . . . . 485
4.7.11 Characteristic Function . . . . . . . . . . . . . . . . . . . . . . . . 489
4.7.12 Central Limit Theorem . . . . . . . . . . . . . . . . . . . . . . . . 493
4.7.13 Univariate Hypothesis and Adequation tests . . . . . . . . . . . . . 499
4.7.14 Robustness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 586
4.7.15 Multivariate Statistics . . . . . . . . . . . . . . . . . . . . . . . . . 630
4.7.16 Propagation of Errors (experimental uncertainty analysis) . . . . . . 667
Sciences.ch 43/3049

Draft
4. Arithmetic Vincent ISOZ [EAME v3.0-2013]
4.1 Proof Theory
W
E have chosed to begin the study of Applied Mathematics by the theory that
seems to us the most fundamental and important in the field of pure and exact
sciences: Proof Theory. The proof theory and of propositional calculus (logic)
has three objectives through this book:
1. Teach to the reader how to reason and demonstrate (prove), and this independently of the
specialization field.
2. Show that the process of a demonstration (proof) is independent of the language used.
3. Prepare the reader to the Logic Theory (see section Logic Systems).
4. Prepare the path to Gödel’s incompleteness theorem (main goal of this section!).
5. Prepare the reader to the Automata Theory (see section Automata Theory).
Gödel’s theorem is probably the most exciting point because if we define religion as a system of
thought that contains unprovable statements, then it contains elements of faith, and Gödel tells
us that mathematics is not only a religion, but that then it is the only religion that can prove it is
one!
Remarks
R1. It is (very) strongly advised to read this section in parallel with those on Automata
Theory and Logical Systems (including Boolean Algebra) available in Theoretical
Computing chapter of this book.
R2. We must approach Proof Theory as a sympathetic curiosity but which basi-
cally brings nothing much except working/reasoning methods. Moreover, its purpose is
not to show that everything is demonstrable but that any proof can be done on a common
language starting from a finite number of rules.
Often when a student arrives in a graduate class he learned how to calculate or use algorithms
but almost only a little or even not at all to reason. For all the reasoning the visual media is a
powerful tool (a picture is worth a thousand words) and people who do not see that in tracing a
given curve or straight line the solution appears or who do not see in space are really penalized.
During high school we already manipulate unknown objects but especially to make calculations
and when we reason about objects represented by letters, we can replace them visually by a real
number, a vector, etc. At a given level we ask people to reason on more abstract structures and
therefore to work on unknown objects which are elements of a set itself unknown, for example
elements of any group (see section Set Theory). This visual support thus doesn’t exist anymore.
We ask so often to students to reason, to demonstrate the properties, but almost no one has ever
taught them to reason properly, writing proofs, control proofs. If we ask a graduate student what
44/3049 Sciences.ch

Draft
is a proof, it most likely he will have some difficulty to answer. He can say that it is a text in
which there are keywords like "therefore", "because", "if", "if and only if", "take a x such that",
"assume", "lemma", "theorem", "let us look for a contradiction", etc. But he will probably be
unable to provide the grammar of these texts nor their basics, and besides, its teachers, if they
have not taken a course in Proof Theory, would probably be unable too.
To understand this situation, remember that to speak a child does not need to know the grammar.
He imitates his surroundings and it works very well: most of time a six year old child know to
use complicated sentences without ever having done grammar. Most teachers also do not know
the grammar of reasoning but, for them, the imitation process has work well and thus they
reason correctly. The experience of the majority of university teachers shows that this process
of imitation works well for very good students, and then it is enough, but it works much less, if
not at all, for many others.
As the complexity level is low (especially during an "equational" type reasoning), grammar
is almost useless but when the level increase or when we do not understand why something
is wrong, it becomes necessary to do some grammar to progress. Teachers and students are
familiar with the following situation: in a school assignment the corrector barred whole page
of a large red line and write "false" in the margin. When the student asks what is wrong, the
corrector can only say things like "this has no relation with the requested proof", "nothing is
right", ..., which help obviously not the student to understand. This is partly because the text
written by the student uses the appropriate words but in a more or less random way and can not
give meaning to the assembly of these words. In addition, the teacher does not have the tools to
explain what is wrong. We must therefore give them to him!
These tools exist but are fairly recent. The proof theory is a branch of mathematical logic whose
origin is the crisis of the foundations: there was a doubt about what we had the "right" to do
in a mathematical reasoning (see the "foundations crisis" further below). Paradoxes appeared,
and it was then necessary to clarify the rules of proof and to verify that these rules are not
contradictory. This theory appeared in the early 20th century, which is very new since most of
the mathematics taught in the first half of the university is known since the 16th-17th century.
4.1.0.1 Foundations Crisis
For the Greeks philosophers geometry was considered the highest form of knowledge, a pow-
erful key to the metaphysical mysteries of the universe. It was rather a mystical belief and the
link between mysticism and religion was made explicit in cults like those of the Pythagoreans.
No culture has been deified a man for discovering a geometrical theorem! Later, mathematics
was regarded as the model of a priori knowledge in the Aristotelian tradition of rationalism.
No culture has since challenged a man for having discovered a geometrical theorem! Later,
mathematics was regarded as the model of a priori knowledge in the Aristotelian tradition of
rationalism.
The astonishment of the Greeks philosophers for mathematics has not left us, we find it in
the traditional metaphor of mathematics as "Queen of Science". It was strengthened by the
spectacular success of mathematical models in science, success that the Greeks (even ignoring
the simple algebra) had not anticipated. Since the discovery by Isaac Newton’s of integral
Sciences.ch 45/3049

Draft
calculus and the inverse square law of gravity in the late 1600s the phenomenal sciences and
higher mathematics remained in close symbiosis - to the point that a predictive mathematical
formalism was became the hallmark of a "hard science".
After Newton, during the next two centuries, science aspired to that kind of rigour and purity
that seemed inherent in mathematics. The metaphysical question seemed simple: mathemat-
ics seemed to have a perfect a priori knowledge, and among all sciences, those that were able
to mathematize most perfectly were the most effective for predicting phenomena. The per-
fect knowledge therefore, was in a mathematical formalism that, once reached by science and
embracing all aspects of reality, could found a posteriori empirical knowledge on an a priori
rational logic. It was in this spirit that Marie Jean-Antoine Nicolas de Caritat, Marquis de
Condorcet (French philosopher and mathematician), undertook to imagine describing the entire
Universe as a set of partial differential equations being solved one after the other.
The first break in this inspiring picture appeared in the second half of the 19th century, when
Riemann and Lobachevsky separately proved that Euclid’s parallel axiom could be replaced
by other geometries that produced "consistent" (we will come back more on this word further
below). Riemannian geometry was modelled on a sphere, these of Lobatschewsky, on rotation
of a hyperboloid.
The impact of this discovery was later obscured by great upheaval, but at the time it made a
thunderclap in the intellectual world. The existence of mutually inconsistent axiomatic systems,
each of which could be a model for the phenomenal Universe, relied entirely into question the
relation between mathematics and theoretical physics.
When we knew only Euclid, there was only one possible geometry. One could believe that the
Euclid’s axiom (see section of Euclidien Geometry) were a kind of knowledge a priori perfect
on the geometry in the phenomenal world. But suddenly we had three geometries, embarrassing
for metaphysical subtleties.
Why would we choose between the axioms of plane geometry, spherical and hyperbolic geom-
etry as real descriptions? Because all three are consistent, we can not choose any a priori as a
foundation - the choice must be empirical, based on their predictive power in a given situation.
Of course, the theoretical physicists have long been accustomed to choose a formalism to study
a scientific problem. But it was already accepted widely, if not unconsciously, that the need to
do so was based on human ignorance, and with logic or good enough mathematics, one could
infer the right choice from principles first, and produce a priori descriptions of reality that had
to be confirmed afterwards by empirical verification.
However, Euclidean geometry, seen for hundreds of years as the model of axiomatic perfection
of mathematics, had been dethroned. If we could not know a priori something as basic as the
geometry in space, what hope was there for a pure rational theory that would encompass all of
nature? Psychologically, Riemann and Lobachevsky had struck at the heart the mathematical
enterprise as it had been designed before.
Moreover, Riemann and Lobachevsky have pushed the nature of mathematical intuition into
question. It was easy to believe implicitly that mathematical intuition was a form of perception
- a way to glimpse the Platonic world behind reality. But with two other geometries pushing the
Euclid one in it’s limit, no one could never be sure to know what the world really looks like.
46/3049 Sciences.ch

Draft
Mathematicians responded to this dual problem with excessive rigour, trying to apply the ax-
iomatic method in all mathematics. In the pre-axiomatic period, the proofs were often based on
commonly accepted intuitions of the "reality" of mathematics, which could not automatically
be regarded as valid.
The new way of thinking about mathematics led to a series of spectacular success. Yet this
had also a price. The axiomatic method made the connection between mathematics and the
phenomenal reality increasingly close. Meanwhile, discoveries suggested that mathematical
axioms that appeared to be consistent with phenomenal experience could lead to dizzying con-
tradictions with this experience.
Most mathematicians quickly became "formalist" arguing that pure mathematics could only
be regarded as a kind of elaborate philosophy game that was played with symbols on paper
(that’s the theory that is behind the mathematical prophetic qualification "zero content system"
by Robert Heinlein). The "Platonic" belief in the reality of mathematical objects, in the old-
fashioned way, seemed good for the trash, despite the fact that mathematicians still feel like
platoniciens during the process of discovery of mathematics.
Philosophically, then, the axiomatic method led most mathematicians to abandon previous be-
liefs in the metaphysical specificity of mathematics. It also produced the contemporary rupture
between pure and Applied Mathematics. Most of the great mathematicians of the early modern
period - Newton, Leibniz, Fourier, Gauss and others - also occupied phenomenal science. The
axiomatic method had hatched the modern idea of the pure mathematician as a great aesthete,
heedless of physics. Ironically, formalism gave the pure mathematicians a bad addiction to
the Platonic attitude. The researchers in Applied Mathematics ceased to meet physicists and
learned to put themselves in their behind.
This takes us to the early 20th century. For the beleaguered minority of Platonists, the worst
was yet to come. Cantor, Frege, Russell and Whitehead showed that all pure mathematics could
be built on the simple foundation of the Set Theory axiomatic. This suited well the formalists:
the mathematics were reunifying, at least in principle, from a small set of rules detached of a
big one. Platoniciens also were satisfied, if a great structure appeared, consistent keystone for
the whole mathematics, the metaphysical specificity of mathematics could still be saved.
In a negative way, though, a Platonist had the last word. Kurt Gödel put his grain of sand in
the program of axiomatization formalism when he proved that any sufficiently powerful axiom
system to include integers numbers had to be either inconsistent (contain contradictions) or
incomplete (too weak to decide the rightness or the falsity of some statements of the system).
And that’s more or less where things stand today. Mathematicians know that many attempts to
advance mathematics as a priori knowledge of the Universe must face numerous paradoxes and
unable to decide which axiom system describes the real mathematics. They have been reduced
to hope that standards axiomatizations are not inconsistent but just incomplete, and wondering
anxiously what contradictions or unprovable theorems are waiting to be discovered elsewhere.
However, on the front of empiricism, mathematics was always a spectacular success as a theo-
retical construction tool. The great success of physics in the 20th century (General Relativity
and Quantum physics) pushed so far out of the realm of physical intuition, they could only be
understood by meditating deeply on their mathematical formalism, and extending their logical
conclusions, even when those findings seemed wildly bizarre. What irony! Just as the mathe-
Sciences.ch 47/3049

Draft
matical perception were to appear always less reliable in pure mathematics, it became more and
more indispensable in phenomenal science.
In contrast to this background, the applicability of mathematics to phenomenal science poses
a more difficult problem than at first appears. The relation between the mathematical models
and prediction of phenomena is complex, not only in practice but also in principle. Even more
complex, as we now know, there are ways to axiomatize mathematics that mutually exclude
themselves!
But why is there only one good choice of mathematical model? That is, why is there a mathe-
matical formalism, for example for quantum physics, so productive that it predicts the discovery
of new observable particles?
To answer this question we will can observe that, as well, works as a kind of definition. For
many phenomenal systems, such exact predictive formalism has not been found, and none seem
plausible. You can easily find such examples: climate or the behavior of a superior economy to
that of a town - systems so chaotically interdependent that exact prediction is actually impossi-
ble (not only in practice but in principle).
48/3049 Sciences.ch

Draft
4.1.1 Paradoxes
Since ancient times, some logicians had noticed the presence of many paradoxes within ratio-
nality. In fact, we can say that despite their number, these paradoxes are merely illustrations
of a few paradoxical structures. Let us look to for general culture to the most famous which
constitute the class of "undecidable propositions".
Example:
The paradox of the class of classes (Russell)
There are two types of classes: those that contain themselves (or reflexive classes:
the class of non-empty sets, the class of classes, ...) and those who do not contains
themselves (or non-reflexives classes: the class of work to be returned, the class of blood
oranges, ...). The question is the following: is the class of non-reflexives classes itself
reflexive or non-reflexive? If it is reflexive, it contains itself and is thus in the class of
non-reflexives classes that it represents, which is contradictory. If it is non-reflexive, it
must be included in the class of non-reflexives classes and becomes ipso facto reflexive,
we are facing again a contradiction.
This Russell’s paradox is often known mainly under the two following variants:
• Does the set of all sets that do not contain themselves contain himself?
The answer is: If Yes, then No and if No then Yes...
• Those who do not shave themselves are shaved by the barber but not those who
shave themselves. So who shaves the barber?
The answer is: If the barber shave himself he enters in the category of peo-
ple that shave themselve so he does not shave himself because he is the barber...
But if he does not shave himself he enters in the category of people that are shaved
by the barber... The answer is also undecidable...
Russell’s paradox challenges the notion of a set as a collection defined by common ownership!
In one shot it destroys the logic (undecidable proposition) and set theory... because the overall
concept of all sets is an impossibility!!! The self-reference is the center of this logical problem!
This paradox also returns to the question whether a math question correctly formulated (logical)
necessarily admits an answer? Said in another way: is any mathematical statement provable...
and it is Gödel that many years after the statement of Russell’s paradox proved mathematically
that the answer is No !!!!!! In other words, there will always be questions unanswered because
any system (living language or mathematical tool) based itself is necessarily incomplete! This
is the famous impact of Gödel’s incompleteness theorem!
Let us see another application of the Russell’s Paradox:
Sciences.ch 49/3049

Draft
Example:
In a library, there are two types of catalogues: Those who mention themselves
and those who does not mention themselves. A librarian must draw up a catalogue of
all catalogues that do not mention themselves. Having completed its work, our librarian
asks whether or not to mention the catalogue that is precisely drafting. At this point, he
is struck perplexity. If he does not mention this catalogue it will be a catalogue that is
not mentioned and which should therefore be included in the list of catalogues that does
not mention themselves. On the other hand, if he mentions the catalogue, this catalogue
will become a catalogue that is mentioned and must therefore not be included in this
catalogue, since it is the catalogue of catalogues which does not mention themselves.
A variations of the previous paradox is the well-known liar paradox:
Example:
Let us provisionally deﬁne lying as the work of making a false proposition. The
Cretan poet Epimenides said: All Cretans are liars, this is the proposal P. How to
decide the truthfulness of P? If P is true, as Epimenides is Cretan, P must be false. P
must therefore be false to be true, which is contradictory.
As would have made understand the logician Ludwig Wittgenstein, these paradoxes ultimately
show that mathematics is a pretty good tool to show the logic but not to talk about it. Give with
mathematics an independent existence to this algebraic entities is madness and it is this that
produces monsters like the set of all the sets... The logic is empty and can not tell the reality, it
restrict to be just a picture of it.
50/3049 Sciences.ch

Draft
4.1.1.1 Hypothetical-Deductive Reasoning
The hypothetical-deductive reasoning is, we know (see the Introduction of the book), the ability
of the learner to deduce conclusions from pure hypotheses and not only of a real observation.
It is a thought process that seeks to identify a causal explanation of any phenomenon (we will
come back on this during our first steps in physics). The learner who uses this type of reasoning
begins with a hypothesis and then tries to prove or disprove his hypothesis following the block
diagram below:
Figure 4.1 – Hypothetical-Deductive Reasoning block diagram
The deductive procedure is to hold as true, provisionally, this first proposal that we name, in
logic a predicate (see further below for more details) and to draw all the consequences logi-
cally necessary, that is to say to look for its implications.
Example:
Consider the proposal P: X is a man, it implies the following proposition Q:
X is mortal.
The expression P ⇒ Q (if it is a human it is necessarily mortal) is a predicative impli-
cation (hence the term predicate). There is no case in this example where we can state
P without Q. This example is that of strict implication, as we find in the syllogism
(logical reasoning figure).
Sciences.ch 51/3049

Draft
Remark
Experts have shown that the hypothetical-deductive reasoning develops gradually by chil-
dren from six to seven years old and that this kind of reasoning is used systematically
starting with a strict propositional function until the age of eleven-twelve.
4.1.2 Propositional Calculus
The propositional calculus (or propositional logic) is an absolutely indispensable prelimi-
nary to tackle a background in science, philosophy, law, politics, economics, etc. This type of
calculation allows for decisions or testing procedures. These help to determine when a logical
expression (proposition) is true and especially if it is always true.
Deﬁnitions:
D1. An expression that is always true whatever the content language of the variables that
compose it is named a valid expression, a tautology or a law of propositional logic.
D2. An expression that is always false is named a contradiction or antilogy.
D3. An expression that is sometimes true, sometimes false is called a contingent expression.
D4. We name assertion an expression that we can say unambiguously whether it is true or
false.
D5. The object language is the language used to write logical expressions.
D6. The meta-language is the language used to talk about the object language in everyday
language.
Remarks
R1. There are expressions that are actually not assertions. For example, the statement
this statement is false is a paradox that can be neither true nor false.
R2. Consider a logical expression A. If it is a tautology, we frequently note it
|= A and the A |= if it is a contradiction.
R3. In mathematics we can try to prove in a general way that an assertion is true,
but not that it is false (if this is the case we give just one example).
52/3049 Sciences.ch

Draft
4.1.2.1 Propositions
Definition: In logic, a proposition is a statement that has meaning. That means we can say
unambiguously whether this statement is true (T) or false (F). This is what we name the Law
of excluded middle.
Example:
I lie is not a proposition. If we assume that this statement is true, it is an affir-
mation of his own disability, so we should conclude that it is false. But if we assume
that it is false, then the author of this statement does not lie, so he told the truth, thus the
proposal would be true...
Definition: A proposition in binary logic (where the proposals are either true or false) is
therefore never true and false at the same time. This is what we call the principle of non-
contradiction.
Thus, a property on the set of propositions E is an application P from E to the set of truth
values True, False {T, F}:
P : E → {T, F} (4.1.1)
We speak about associated subset , when the proposition only generates a portion E of E and
vice versa.
Example:
I lie is not a proposition. If we assume that this statement is true, it is an affir-
mation of his own disability, so we should conclude that it is false. But if we assume
that it is false, then the author of this statement does not lie, so he told the truth, thus the
proposal would be true...
In E = N, if the proposition P(x) is x is even, then {0, 2, 4, ..., 2k, ...} which is effectively
an associated subset of E but with same Cardinal (section Set Theory).
Definition: Let P be a property of the set E. A property Q on E is a negation of P if and
only if, for any x ∈ E:
• Q(x) is F (false) if P(x) is T (true)
• Q(x) is T (true) if P(x) is F (false)
We can gather these conditions in a table called truth table:
Sciences.ch 53/3049

Draft
P Q
T F
F T
Table 4.1 – Truth table of values
Table that we can also ﬁnd or also write in the most explicit following form:
P Q
True False
False True
Table 4.2 – Truth table of explicit values
or in binary form:
P Q
1 0
0 1
Table 4.3 – Truth table of binary values
In other words, P and Q always have opposite truth values. We denote this kind of statement
Q is a negation of P:
Q ⇔ ¬P (4.1.2)
where the symbol ¬ is the negation connector.
Remark
The expressions must be well-formed expressions (often abbreviated WFE). By deﬁ-
nition, any variable is a well-formed expression, thus ¬P is a well-formed expression. If
P, Q are well-formed formulas, then P ⇒ Q is a well-formed expression (the expression
I am lying is not well-formed because it contradicts itself).
54/3049 Sciences.ch

Draft
4.1.2.2 Connectors
There are other types of logical connectors:
Definition: Let P and Q two properties set defined on the same set E. P ∨ Q (read P y‚ Q)
is a property on E defined by:
• P ∨ Q is true if at least P or Q are true
• P ∨ Q is false otherwise
We can create the truth table of the y‚ connector or disjunction connector ∨:
P Q P ∨ Q
T T T
T F T
F T T
F F F
Table 4.4 – OR truth table
It should be easy to convince yourself that if the parts P, Q of E à are respectively associated
with the properties P, Q thus P ∪ Q (see section Set Theory) is associated to P ∨ Q.
P ↔ P
Q ↔ Q
P ∨ Q ↔ P ∪ Q
(4.1.3)
The connector ∨ is associative (no doubt about the fact that it is commutative!). For proof, just
do a truth table where you can check that:
[P ∨ (Q ∨ R)] = [(P ∨ Q) ∨ R] (4.1.4)
Definition: There is also the exh connector or also named conjunction connector ∧ for
whatever are P, Q two properties defined on E, P ∧ Q is a property defined on E by:
• P ∧ Q is true if both properties P, Q are true (the famous syllogism: All men are mortal,
Socrates is a man, therefore Socrates is mortal is a famous example).
• P ∧ Q if false otherwise
We can create the truth table of the exh connector or disjunction connector ∨:
P Q P ∨ Q
T T T
T F F
F T F
F F F
Table 4.5 – AND truth table
Sciences.ch 55/3049

Draft
It should be also almost easy to convince yourself that if the parts P, Q of E à are respectively
associated with the properties P, Q thus P ∩ Q (see section Set Theory) is associated to P ∧ Q.
P ↔ P
Q ↔ Q
P ∧ Q ↔ P ∩ Q
(4.1.5)
The connector ∧ is associative (no doubt about the fact that it is commutative!). For proof, just
do a truth table where you can check that:
[P ∧ (Q ∧ R)] = [(P ∧ Q) ∧ R] (4.1.6)
The connectors ∨, ∧ are distributive one on the other. Using a simple truth table, we can show
that (ask me if you want a put the truth table):
[P ∨ (Q ∧ R)] = [(P ∨ Q) ∧ (P ∨ R)] (4.1.7)
as well as:
[P ∧ (Q ∨ R)] = [(P ∧ Q) ∨ (P ∧ R)] (4.1.8)
Deﬁnition: The negation operator ¬ transform a True value into a False value such that:
¬T = F (4.1.9)
¬F = T (4.1.10)
So in logic, negation, also named logical complement, is an operation that takes a proposition
P to another proposition not P, written ¬P or sometimes ¯P, which is interpreted intuitively
as being True when P is false and False when P is True. Negation is thus a unary (single-
argument) logical connective.
As we will prove it in detail in the section of Logic System (using a simple truth table) the De
Morgan’s laws provide a way of distributing negation over disjunction and conjunction:
¬(P ∧ Q) ⇔ [(¬P) ∨ (¬Q)] (4.1.11)
¬(P ∨ Q) ⇔ [(¬P) ∧ (¬Q)] (4.1.12)
(4.1.13)
Remark
To see the details of all logical operators, the reader should read the section of Logical
Systems (see chapter Theoretical Computing) where the identity, the double negative,
the idempotence, associativity, the distributive properties, the De Morgan relations are
presented more formally and with full details.
Let us now come back on the logical implication connector sometimes also named just the
conditional denoted by the symbol ⇒.
56/3049 Sciences.ch

Draft
Remark
In some books on propositional calculus, this connector is denoted by the symbol ⊃ and
as part of the proof theory we often prefer the symbol →.
Let P, Q two properties given on E. P ⇒ Q is a property on E deﬁned by:
P1. P ⇒ Q is False if P is True and Q is False.
P2. P ⇒ Q is True otherwise.
In other words, the fact that P logically implies Q means that Q is True for any assessment for
which P is True. The implication is therefore the famous if... then ....
If we write the truth table of the implication (caution with the before last line!!!):
P Q P ⇒ Q
T T T
T F F
F T T
F F T
Table 4.6 – Implication truth table
In other terms, a False proposition implies that any conclusion will always be True. If the
proposition is True the implication can be True only if the result is True.
Example:
Consider the proposition: If you get your diploma, I buy you a computer.
Of all cases, only one corresponds to a broken promise: the one where the child
graduates, and still has no computer (second line in the table above).
What means exactly this promise, that we will write as following:
You have your degree ⇒ I buy you a computer?
Exactly this:
• If you have get graduate, for sure, I will buy you a computer (I can not not buy it).
• If you do not get graduate, I said nothing.
The implication gives us that from any false proposition we can deduce any proposal (last two
lines).
Sciences.ch 57/3049

Draft
Example:
In a course teached by Russell on the subject from a false proposition, any pro-
posal can be inferred, a student asked him the following question (anecdote or legend?):
• Are you saying that from the proposition 2 + 2 = 5, it follows that you are the
Pope?.
• Yes, answered Russell.
• And could you prove it!, asked the student skeptical...
• Certainly, answered Russell, who immediately offered the following proof:
1. Suppose that 2 + 2 = 5.
2. Subtract 3 from each member of the equality, we thus get 1 = 2.
3. By symmetry 2 = 1.
4. The Pope and I are 2. Since 2 = 1, Pope and I are 1. It follow I’m the Pope.
The implication connector is essential in mathematics, philosophy, etc. It is a backbone of any
proof, evidence or deduction. It has the following useful properties (normally easy to check
with a small truth table):
P ⇒ Q ⇔ [(¬Q) ⇒ (¬P)] (4.1.14)
P ⇒ Q ⇔ [(¬P) ∨ Q] (4.1.15)
And we have from the last property (again verifiable by a truth table):
¬(P ⇒ Q) ⇔ [P ∧ (¬Q)] (4.1.16)
The logical equivalence connector or biconditional connector denoted most of times by ⇔
or sometimes by ↔ meaning by definition:
(P ⇔ Q) ⇔ [(P ⇒ Q) ∧ (Q ⇒ P)] (4.1.17)
in other words, the first expression has the same value for all evaluation of the second. It is
the same with the following relation that is more atomic as the logical equivalence is reduced
only to the use of ∧, ∨ and negation ¬ (combination of what we have seen above):
(P ⇔ Q) ⇔ [(P ∧ Q) ∨ (¬P ∧ ¬Q)] (4.1.18)
When we prove such equivalence of two expressions we can therefore say that: we prove that
the equivalence is a tautology.
The truth table of the equivalence is logically given by:
P Q P ⇒ Q Q ⇒ P Q ⇔ P
T T T T T
T F F T F
F T T F F
F F T T T
Table 4.7 – Truth table of the equivalence
58/3049 Sciences.ch

Draft
P ⇔ Q means (when its true!) that P and Q always have the same truth value or P and Q
are equivalent. This is True if P and Q have the same value, False otherwise.
Of course (it is a tautology):
¬(P ⇔ Q) ⇒ (P ⇒ ¬Q) (4.1.19)
The relation P ⇔ Q is equivalent to that P is a necessary and sufficient condition for Q and
that Q is a necessary and sufficient condition for P.
The conclusion is that the conditions of the types: necessary, sufficient, necessary and
sufficient can be reformulated with the terms only if, if, if and only if.
Therefore:
1. P ⇒ Q reflects the fact that Q is a necessary condition for P or in other words, P is True
only if Q is True (in the truth table, when P ⇒ Q is equal to 1 we see that P is 1 only if
Q is also 1). We also say that if P is true then Q is true.
2. P ⇐= Q or what is the same Q ⇒ P reflects the fact that Q is a sufficient condition for
P or in other words, P is True if Q is True (in the truth table, when Q ⇒ P takes the
value 1 we see that P is 1 if Q is 1 too).
3. P ⇔ Q reflects the fact that Q is a necessary and sufficient condition for P or in other
words, P is True if and only if Q is True (in the truth table, when P ⇔ Q takes the value
1 we see that P is 1 if Q is 1 and if and only if Q is equal to 1).
Remark
The expression if and only if therefore corresponds to a logical equivalence and can
only be used to describe a bi-implication !!
The first stage of propositional calculus is the formalization of natural language statements. To
make this work, the propositional calculus finally provides three types of tools:
1. The propositional variables (P, Q, R, ...) symbolize any simple proposals. If the same
variable occurs multiple times, each time it symbolizes the same proposal.
2. The five logical operators: ¬, ∧, ∨, ⇔, ⇒.
3. Punctuation are reduced to only opening and closing parentheses that organize reading so
as to avoid ambiguity.
Sciences.ch 59/3049

Draft
Description Symbol Usage
Negation is an operator that act only one one proposal; it is unary or
monadic. It is not raining will be written ¬P. This statement is true
if and only if P is false (in this case if it is false that it is raining). The
conventional use of negation is characterized by the double negation
law: ¬¬P is equivalent to P.
¬ ¬P
The conjunction or logical product is a binary operator; it connects
two proposals. Every man is mortal AND my car loses oil is written
P ∧ Q. This latter expression is true if and only if P is true and Q is
true.
∧ P ∧ Q
The disconnection or logical sum is also a binary operator. P ∨ Q
is true if and only if P is true OR Q is true or both are true. We can
understand the OR into two ways: either inclusively or exclusively. In
the first first case P ∨ Q is true if P is true, if Q is true or if P and Q
are both true. In the second case, P ∨ Q is true if P is true or if Q is
true but not if both are. The disjunction of propositional calculus is the
inclusive OR and the give to the exclusive OR, that is to say the XOR,
the name of alternative.
∨ P ∨ Q
The implication is also a binary operator. It corresponds roughly to
the linguistic pattern If ... then .... If I have time, I will go see movie
will be written P ⇒ Q. This latter relation is false if P is true and
Q is false. If the result (here Q) is true, the implication P ⇒ Q is
true. When the antecedent (here P) is false, the implication is always
true. This latter remark can be understood if one refers to statements of
the type: If we could put Paris in a bottle, the Eiffel Tower would be
used as a plug. In summary, an implication is false if and only if its
antecedent is true and its consequent is false.
⇒ P ⇒ Q
The bi-implication or equivalence ⇒ is, too, a binary operator: it
symbolizes the terms ... if and only if ... and ... is equivalent to
.... The equivalence of two propositions is true if they have the same
truth value. The bi-implication therefore expressed as a form of identity,
which is why it is often used in definitions.
⇔ P ⇔ Q
Table 4.8 – Summary of logical core operators
The reader should find sometimes by some authors that like to use at minimum the natural
language in their books the symbol the sign ∴ that is sometimes placed before a logical con-
sequence, such as the conclusion of a syllogism. The symbol consists of three dots placed in
an upright triangle and is read therefore. We can also make use of the symbol and is read
because.
Example:
All men are mortal.
Socrates is a man.
∴ Socrates is mortal
60/3049 Sciences.ch

Draft
In this book we will avoid using this notation as the engineers don’t make use of this a lot.
It is possible to establish equivalences between these operators. We have already seen how
the biconditional could be deﬁned as a product of reciprocal conditional, let us see now other
equivalences:
(P ⇒ Q) ⇔ ¬(P ∧ ¬Q)
(P ⇒ Q) ⇔ (¬P ∨ Q)
(P ∨ Q) ⇔ (¬P ⇒ Q)
(P ∧ Q) ⇔ ¬(P ⇒ ¬Q)
(4.1.20)
Remark
The classical operators ∧, ∨, ⇔ can therefore be deﬁned using the canonical operators
¬, ⇒ through equivalence laws between them.
Also notice the two relations of De Morgan (see sectionof Boolean Algebra for the proof):
¬(P ∨ Q) ⇔ (¬P ∧ ¬Q)
¬(P ∧ Q) ⇔ (¬P ∨ ¬Q)
(4.1.21)
They allow to transform the disjunction into conjunction and vice versa:
(P ∨ Q) ⇔ ¬(¬P ∧ ¬Q)
(P ∧ Q) ⇔ ¬(¬P ∨ ¬Q)
(4.1.22)
4.1.2.3 Decision procedures
We have previously introduced the basic elements allowing us to operate on expressions from
properties (propositional variables) without saying much about the handling of such expres-
sions. So now you need to know that in propositional calculus there are two ways to establish
that a proposition is a law of propositional logic. We can either:
1. Use non-automated procedures
2. Use axiomatic and demonstrative procedures
Remark
In many books these procedures are presented before the structure of the propositional
language. We chose here to do the opposite thinking that the approach would be easier.
Sciences.ch 61/3049

Draft
4.1.2.3.1 Non-axiomatic procedural decisions
Several of these methods exist, but we will limit ourselves here to the simplest of them,
that of the matrix calculation, often referred to as methods of truth tables.
The construction procedure is as we have already seen quite simple. Indeed, the truth value of a
complex expression is a function of the truth value of the simple statements that compose it, and
finally based on the truth value of propositional variables that makes it. Considering all possible
combinations of truth values of propositional variables, we can determine the truth values of the
complex expression.
Truth tables, as we have seen it, give us the possibility to decide, about any proposition, if
this latter is a tautology (always true), a contradiction (always false) or a contingent expression
(sometimes true, sometimes false).
We can distinguish at least four ways to combine propositional variables, brackets and connec-
tors:
Name Description Example
1 Malformed statement Nonsense. Neither true nor false (∨P)Q
2 Tautology Statement always true P ∨ ¬Q
3 Contradiction Statement always false P ∧ ¬P
4 Contingent statetement Statement sometimes true, sometimes false P ∨ Q
Table 4.9 – Combination of propositional variables
The method of truth tables helps to determine the type of expression that are well-formed to
which we have to face. It requires, in principle, no invention, it is only a mechanical pro-
cedure. Axiomatized procedures, however, are not entirely mechanical. Inventing a proof as
part of an axiomatized system requires sometimes hability, practice or luck. Regarding to truth
tables, here is the protocol to follow:
When facing a well-formed expression, or function of truth, we first determine how many dis-
tinct propositional variables we are dealing with. We then examine the various arguments that
form this expression. We then construct a table with 2n
columns (n being the number of vari-
ables and without forgotting that they are binary variables!) and a number of columns equal to
the number of arguments plus columns for the expression itself and its other components (see
previous examples). Then we assign to the variables the various combinations of True (1) and
False (0) values that may be conferred upon them. Each row corresponds to a possible outcome
and all of the rows is the set of all possible outcomes. There is, for example, a possible outcome
wherein P is a true statement while Q is false.
4.1.2.3.2 Axiomatic procedural decisions
The axiomatization of a theory implies, besides its formalization, that we start form a
finite number of axioms and that through the controlled transformation of these, we can get
all the theorems of this theory. So we start from a few axioms whose truth is a statement (not
proven). We determine afterwards deduction rules for manipulating the axioms or expression
62/3049 Sciences.ch

Draft
obtained from these. The sequence of these deductions is a proof that leads to a theorem, a law
or lemma.
We will now briefly present two axiomatic systems, each consisting of axioms using two specific
rules named inference rules (intuitive rules):
1. The modus ponens: If we prove A and A ⇒ B, then we can deduce B. A is named the
minor premise and B the major premise of the modus ponens rule.
Example:
From:
x y (4.1.23)
and:
(x y) ⇒ (y x) (4.1.24)
we can deduce that:
y x (4.1.25)
2. The substitution: we can in a schema of axioms replace a letter by any formula, at the
condition that all identical letters are replaced by identical formulas!
Let us give as an example, two axiomatic systems: the axiomatic system of Whitehead
and Russell, the axiomatic system of Lukasiewicz.
(a) The axiomatic system of Russell and Whitehead adopts ¬, ∨ as primitive symbols
and define ⇒, ∧, ⇔ from these latter as follows (easily verifiable relations with truth
tables):
(A ⇒ B) ⇔ ¬A ∨ B
(A ∧ B) ⇔ ¬(¬A ∨ ¬B)
(A ⇔ B) ⇔ (A ⇒ B) ⇒ (B ⇒ A)
(4.1.26)
This system includes 5 axioms, somewhat quite obvious plus two rules of inference.
Axioms are given here using non-primitive symbols, as did Whitehead and Russell:
A1. (A ∨ A) ⇒ A
A2. B ⇒ (A ∨ B)
A3. (A ∨ B) ⇒ (B ∨ A)
A4. (A ∨ (B ∨ C)) ⇒ (B ∨ (A ∨ C))
A5. (B ⇒ C) ⇒ (A ∨ C)
we have already presented above some of these.
Sciences.ch 63/3049

Draft
Remark
These five axioms are not independent of each other. The fourth can be ob-
tained from the other four.
For example, to justify that ¬A ∨ A has a sense, we can proceed as following:
(1) B ⇒ (A ∨ B) Axiom A2
(2) B ⇒ (A ∨ A) (1) and substitution
(3) (B ⇒ C) ⇒ ((A ∨ B) ⇒ (A ∨ C)) ⇒ (A ∨ C)) Axiom A5 (necessary)
(4) (B ⇒ C) ⇒ ((¬A ∨ B) ⇒ (¬A ∨ C)) (3) and substitution
(5) (B ⇒ C) ⇒ ((A ⇒ B) ⇒ (A ⇒ C)) (4) and property of ⇒
(6) ((A ∨ A) ⇒ A) ⇒ ((A ⇒ (A ∨ A)) ⇒ (A ⇒ A)) (5) and substitution
(7) (A ⇒ (A ∨ A)) ⇒ (A ⇒ A) (6) (modus ponens)
(8) (A ⇒ A) ⇒ (A ⇒ A) (7) and axiom A1
(9) A ⇒ A (8) and modus ponens
(10) ¬A ∨ A (9) and property of ⇒
(b) The axiomatic system of Lukasiewicz includes three axioms, plus the two rules of
inference (modus ponens and substitution):
A1. (A ⇒ B)((B ⇒ C) ⇒ (A ⇒ C))
A2. A ⇒ (¬A ⇒ B)
A3. (¬A ⇒ A) ⇒ A
Here is the proof of the first two axioms in the system of Russell and Whitehead.
These are the formulas (6) and (16) of the following derivation:
(1) (A ∨ (B ∨ C)) ⇒ (B ∨ (A ∨ C)) Axiom A5
(2)
(¬(B ⇒ C)∨(¬(A∨B)∨(A∨C)))
⇒ (¬(A∨B)∨(¬(B ⇒)∨(A∨C)))
(1) and substitution
(3) ¬(A ∨ B) ⇒ ((B ⇒ C) ⇒ (A ∨ C)) A4 on (2) and modus ponens
(4) (A ∨ B) ⇒ ((B ⇒ C) ⇒ (A ∨ C)) (3) by the property of ⇒
(5) (¬A ∨ B) ⇒ ((B ⇒ C) ⇒ (¬A ∨ C)) (4) and substitution
(6) (A ⇒ B) ⇒ ((B ⇒ C) ⇒ (A ⇒ C)) (5) and property of ⇒
(7) (B ⇒ (A ∨ B)) ⇒
((A ∨ B) ⇒ (B ∨ A))
⇒ (B ⇒ (B ∨ A))
(6) and substitution
(8) ((A ∨ B) ⇒ (B ∨ A)) ⇒ (B ⇒ (B ∨ A)) (7) modus ponens
(9) B ⇒ (B ∨ A) (8) modus ponens
(10) ¬B ⇒ (¬B ∨ A) (9) and substitution
(11) ¬¬B ∨ (¬B ∨ A) (10) and property of ⇒
(12) ¬¬B ∨ (¬B ∨ A) ⇒ (¬B ∨ (¬¬B ∨ A)) A4 + substitution with (11)
(13) ¬B ∨ (¬¬B ∨ A) (12) and modus ponens
(14) B ⇒ (¬¬B ∨ A) (19) and property of ⇒
(15) B ⇒ (¬B ∨ A) (14) and property of ⇒
(16) A ⇒ (¬A ⇒ B) (15) and substitution
These axiomatizations let us found as theorems all tautologies or laws of the propositional logic.
From everything that has been said so far, we can try to define what is a proof!!!!
64/3049 Sciences.ch

Draft
Definition: A finite sequence of formulas B1, B2, . . . , Bm is name a proof from the assump-
tions/hypothesis A1, A2, . . . , An if for each i:
• Bi is one of the assumptions/hypothesis A1, A2, . . . , An
• or Bi is a variant of an axiom
• or Bi is inferred (by the application of the modus ponens rule) from the major premise
Bj and minor premise Bk where j, k i
• or Bi is inferred (by the application of the substitution rule) from an anterior premise Bj,
the replaced variable not appearing in A1, A2, . . . , An
Such a sequence of formulas, Bm being the final formula of the sequence, is more explicitly
named proof of Bm from the assumptions/hypothesis (or axioms) A1, A2, . . . , An, what we
also write:
A1, A2, . . . , An Bm (4.1.27)
More explicitly a proof is a deductive argument for a mathematical statement. In the argument,
other previously established statements, such as theorems, can be used. In principle, a proof
can be traced back to self-evident or assumed statements, known as axioms.
Proofs employ logic but usually include some amount of natural language which usually admits
some ambiguity. In fact, the vast majority of proofs in written mathematics can be considered
as applications of rigorous informal logic. Purely formal proofs, written in symbolic language
instead of natural language, are considered in proof theory.
Remark
Note that when we try to prove a result from a number of assumptions, we do not try to
prove the assumptions themselves!
Sciences.ch 65/3049

Draft
4.1.2.4 Quantifiers
We have to complete the use of the connectors of propositional calculus by what we name
quantifiers if we wish to solve some problems. Indeed, the propositional calculus does not
allow us to state general things about the elements of a set, for example. In that sense, propo-
sitional logic is only part of the reasoning. The calculus of predicates on the contrary allows
to formally handle statements such as there exists an x such that [x has an American car] or
for all x [if x is a dachshund, then x is small]. In short, we extend the composed formulas
in order to assert existential quantifiers ( there...) and universal quantifiers ( for every...).
The examples we just gave involve a bit special proposals like x has an American car. This is
proposition with a variable. These proposals are in fact the application of a function to x. This
function, is this that associates x has an American car with x. We will denote this function
by ... has an American car and we say that is a propositional function because it is a function
whose value is a proposal. Or a predicate as we already know.
The existential and universal quantifiers go hand in hand with the use of propositional functions.
The predicate calculus is however limited in the existential and universal formulas. Thus, we
prohibit ourselves to use sentences like there is an affirmation of x such that .... In fact, we
allow ourselves to quantify only individuals. This is why predicate logic is named first-order
logic because it uses variables as basic mathematical objects (while in the second-order logic
they can also be sets).
Before turning to the study of the predicate calculus we define:
D1. The universal quantifier: ∀ (for all)
D2. The existential quantifier: ∃ (exists)
Example:
If any complex number is the product of a non-negative number and a number of
modulus 1 we will write:
∀z ∈ C, ∃p ∈ Z ≥ 0, ∃u ∈ C : (|u| = 1 ∧ z = pu) (4.1.28)
The order of quantifiers is critical to meaning, as is illustrated by the following two propositions:
For every natural number n, there exists a natural number s such that s = n2
.
This is clearly true! It just asserts that every natural number has a square. The meaning of the
assertion in which the quantifiers are turned around is different:
There exists a natural number s such that for every natural number n, s = n2
.
66/3049 Sciences.ch

Draft
This is clearly false! It asserts that there is a single natural number s that is at the same time the
square of every natural number.
A frequent question in physics and mathematics is to know if the universal quantifier has to be
before of after the predicates they refer to. In fact, strictly in terms of formal logic, quantifiers
are always at the beginning of any formula. However, almost no one gives a proof that is written
in the formal language. Even simple proofs would be very long and unreadable. But anyone,
regardless of what natural language they speak, will interpret a sentence in the formal language
in the same way. The price for this clarity of course is readability. Natural languages, because
of their inherent ambiguity, are subject to many more limitations.
Obviously the proper usage of a formal notation or of a more informal one depends particularly
on the context of presentation. It is essential to whom we communicate an idea and this should
guide us to use a suitable level of formal notation.
We use the sometimes the symbol ∃! to say briefly: there is one and only one.
Example:
A famous example is the way to explicit that the logarithm is a bijective function:
∀x ∈ R+
, ∃!y ∈ R∗
+ : x = ln(y) (4.1.29)
We will see now that the Proof Theory and Set Theory is the exact transcription of the principles
and results of Logic (the one with a capitalized L )
Sciences.ch 67/3049

Draft
4.1.3 Predicate Calculus
In mathematics courses (algebra, analysis, geometry, ...), we prove the properties of different
object types (integer, real, matrices, sequences, continuous functions, curves, triangles, ...) . To
prove these properties, we need of course that the objects on which we work are clearly defined
(what is a set, what is a real, what is point, ...?).
In first-order logic and, in particular, in proof theory, the objects we study are the formulas and
proofs. We must therefore give a precise definition of what are these objects. The terms and
formulas are the grammar of a language, oversimplified and calculated exactly to say what we
want without ambiguity and without unnecessary detour.
4.1.3.1 Grammar
Grammar:
D1. The terms designate items for which we want to prove some properties (we will dis-
cussed the latter much more in details further below):
• In algebra, the terms refer to the elements of a set (group, ring, field, vector space,
etc.). We also manipulate sets of objects (subgroups, subrings, subfields, etc.). The
terms which will designate the objects are named second-order terms.
• In analysis, the terms refer most of time to real numbers (for example, if we put
ourselves in functional spaces) or functions.
D2. The Formulas, are the properties of objects we study (we will discussed the latter also
much more in details further below):
• In algebra, we can write formulas to express that two elements commute, that a
subspace is of dimension 3, etc.
• In analysis, we will write formulas to express the continuity of a function, the con-
vergence of a sequence, etc.
• In set theory, formulas can express inclusion of two sets, membership of an element
in a set, etc.
D3. The proof, enable to check if a formula is true. The precise meaning of this word will
also need to be defined. More precisely, they are deductions under assumptions, they
allow to lead from truth to truth, the question of the truth of the conclusion is then
returned to that of the hypothesis, which does not look at the logic but is based on the
knowledge we have on things we talk about.
68/3049 Sciences.ch

Draft
4.1.3.2 Language
In mathematics we use, depending on the area, different languages that are distinguished by the
symbols used. The definition below simply expresses that it is sufficient to only have to give
the list of symbols to specify the language.
Definition: A language is the content of a family (not necessarily finite) of symbols. We
distinguish three kinds of languages: symbols, terms and formulas.
Remarks
R1. We use sometimes the word vocabulary or signature instead of the word
language.
R2. We already know that the word predicate is used instead of the word relation.
We speak then of predicate calculus instead of first-order logic).
4.1.3.2.1 Symbols
There are different types of symbols we will try to define:
D1. The constant symbols (see note below)
Example:
The neutral element n in Set Theory (see section Set Theory)
D2. The function symbols or functors. To each function symbol is assigned a strictly
positive integer that we name her ary: this is the number of arguments of the function
arguments. If the arity is 1 (resp. 2, . . . , n), we say then that the function is unary (resp
binary., ..., n-ary).
Example:
The binary functor of multiplicaton × or · in group theory (see section Set
Theory)
D3. The relation symbol. Similarly to the previous definition, every relation symbol is
associated with a positive or null integer (its arity) that corresponds to its number of
arguments and we talk then of unary, binary, n-ary relation.
Example:
The relation = is a binary relation (see section Set Theory)
Sciences.ch 69/3049

Draft
D4. The individual variables. In what will follow we will give us an infinite set V of vari-
ables. The variables will be recorded as it is traditional by the latin lowercase letters: x,
y, z (possibly indexed: x1, x2, x3).
D5. To this we should add the connectors ¬, ⇒, ∨, ∧ and quantifiers ∀, ∃, ∃! that we exten-
sively discussed above, on which it is now useless to return.
Remarks
R1. A constant symbol can be seen as a function symbol with 0 argument (arity zero).
R2. We consider (unless otherwise stated) that each language contains the binary
relation symbol = (read equal) and the relation symbol with zero argument denoted ⊥
(read bottom or absurd) representing as we already know the value FALSE. In the
description of a language, we will often omit to mention them. The symbol ⊥ is often re-
dundant. We can indeed, without using it, write a formula that is always false. However,
it can represent the FALSE in a canonical way and therefore to write general proofs rules.
R3. The role of functions and relations is very different. As we will see, the function
symbols are used to construct the terms (of language objects) and the relation symbols to
build formulas (the properties of these objects).
4.1.3.2.2 Terms
The terms, we also say the first order terms, are the objects associated with the lan-
guage.
D1. Given L a language, the set T of terms on L is the smallest set containing the variables,
the constants and stable (we do not go out of the set) by applying function symbols of L
to the terms.
D2. A closed term is a term that does not contain variables (by extension, only constants).
D3. For a more formal definition, we can write:
T0 = {t} (4.1.30)
where t is a variable or constant symbol and, for any k ∈ N:
Tk+1 = Tk ∪ {f(t1, . . . , tn)|ti ∈ Tk} (4.1.31)
where f is obviously a function of arity n (let us recall that the arity is the number of
function arguments). Thus, for each arity, there is a degree of set of terms. We have
finally:
T =
k∈N
Tk (4.1.32)
70/3049 Sciences.ch

Draft
D5 . We name height of a term t the smallest k such that t ∈ Tk.
This definition means that variables and constants are terms and that if f is a n-ary func-
tion symbol and t1, . . . , tn are terms then f(t1, . . . , tn) is also a term itself. The set T of
terms is defined by the grammar:
T = {V |Sc|Sf (T , . . . , T } (4.1.33)
This expression must be read as follows: a element of the set T we are defining is either
an element of V (variable) or an element of Sc (the set of symbols of constant), or the
application of a n-ary function symbol f ∈ Sf (constants or variable) of T .
Caution: The fact that f is of the good arity is only implicit in this notation. Moreover,
writing Sf (T , . . . , T ) does not mean that all function arguments are identical, but simply
that these arguments are elements of T .
Remark
It is often convenient to see a term (expression) as a tree, where each node is
labeled with a function symbol (operator or function) and each sheet by a variable
or constant.
In what follows, we will almost always define concepts (or prove results) by recurrence on
the structure or the size of a term.
Definitions:
D1. To prove a property P on the terms, it suffices to prove P for the variables and con-
stants and to prove P(f(t1, . . . , tn)) from P(t1), . . . , P(tn). We do then here a proof by
induction on the height of a term. It is a technique that we will find in the following
chapters.
Mathematical induction as an inference rule can be formalized as a second-order axiom.
The axiom of induction is, in logical symbols:
∀P. [[P(0) ∧ ∀(k ∈ N). [P(k) ⇒ P(k + 1)]] ⇒ ∀(n ∈ N). P(n)] (4.1.34)
In words, the basis P(0) and the inductive step (namely, that the inductive hypothesis
P(k) implies P(k + 1)) together imply that P(n) for any natural number n. The axiom
of induction asserts that the validity of inferring that P(n) holds for any natural number
n from the basis and the inductive step.
Induction can be compared to falling dominoes: whenever one domino falls, the next one
also falls. The first step, proving that P(1) is true, starts the infinite chain reaction.
Sciences.ch 71/3049

Draft
D2. To define a function Φ based on the terms, it is enough to define it on the variables and
constants and tell how we get Φ(f(t1, . . . , tn)) from Φ(t1), . . . , Φ(tn). We do here again
a definition by induction on the height of a term.
Example:
The size (we also say the length) of a term t (size denoted τ(t)) is the number
of function symbols occurring in t. formally:
• τ(x) = τ(c) = 0 if x is a variable and c is a constant.
• τ(f(t1, . . . , tn)) = 1 + i≤i≤n τ(ti)
where the 1 in the last relation represents the term f itself.
4.1.3.2.3 Formulas
Definition: A well-formed formula (WFF) , often simply formula is a word (i.e. a
finite sequence of symbols from a given alphabet) that is part of a formal language. A formal
language can be considered to be identical to the set containing all and only its formulas.
The formulas of propositional calculus, also named propositional formulas, are expressions
such as (A ∧ (B ∨ C)).
An atomic formula is a formula that contains no logical connectives nor quantifiers, or equiva-
lently a formula that has no strict subformulas. The precise form of atomic formulas depends on
the formal system under consideration; for propositional logic, for example, the atomic formu-
las are the propositional variables. For predicate logic, the atoms are predicate symbols together
with their arguments, each argument being a term.
I0 = (t) (4.1.35)
Ik+1 = Ik ∪ (f(t1, . . . , tn)/ti ∈ Ik) (4.1.36)
I = ∪
k∈N
Ik (4.1.37)
I = V |SC|SF (I, . . . , I) (4.1.38)
72/3049 Sciences.ch

Draft
Atom = SR(I, . . . , I) (4.1.39)
F = Atom|F ∨ F|F ∧ F|F → F|F¬F|∃xF|∀xF (4.1.40)
CP = VP |⊥|¬CP |CP ∨ CP |CP ∧ CP |CP → CP (4.1.41)
Γ A → B Γ A
Γ B
→e (4.1.42)
Γ, A A
ax (4.1.43)
Γ A
Γ, B A
aff (4.1.44)
Γ, A B
Γ B → A
→i (4.1.45)
Γ A → BΓ A
Γ B
→e (4.1.46)
Γ AΓ B
Γ A ∧ B
∧i (4.1.47)
Γ A ∧ B
Γ A
∧g
e (4.1.48)
Γ A ∧ B
Γ B
∧d
e (4.1.49)
Sciences.ch 73/3049

Draft
Γ A
Γ A ∨ B
∧g
i (4.1.50)
Γ B
ΓA ∨ B
∨d
i (4.1.51)
Γ A ∨ BΓ, A CΓ, B C
Γ C
∨e (4.1.52)
Γ, A ⊥
Γ ¬A
¬i (4.1.53)
Γ ¬A Γ A
Γ ⊥
¬e (4.1.54)
Γ, ¬A ⊥
Γ A
⊥c (4.1.55)
Γ A x n’est pas libre dans les formules de Γ
∀xA
∀i (4.1.56)
Γ ∀xA
Γ A[x := t]
∀e (4.1.57)
Γ A[x := t]
Γ ∃xA
∃i (4.1.58)
Γ ∃xA Γ, A C x n’est libre ni dans les formules de Γ, ni dans C
Γ C
∃e (4.1.59)
Γ t = t
=i (4.1.60)
74/3049 Sciences.ch

Draft
Γ A[x := t] Γ t = u
Γ A[x := u]
=e (4.1.61)
x1 = x2 x1 = x1
=i
x1 = x2 x1 = x2
ax
x1 = x2 x2 = x1
x1 = x2 → x2 = x1
→i
∀x1, x2(x1 = x2 → x2 = x1
∀i × 2
=e (4.1.62)
F x1 = x2 ∧ x2 = x3
ax
F x1 = x2
∧g
e
F x1 = x2 ∧ x2 = x3
ax
F x2 = x3
∧d
e
F x1 = x3
(x1 = x2 ∧ x2 = x3) → x1 = x3
→i
∀x1, x2, x3((x1 = x2 ∧ x2 = x3) → x1 = x3)
∀i × 3
=e (4.1.63)
∀x, y f(x) = f(y) → x = y (4.1.64)
∀y, ∃x f(x) = y (4.1.65)
Inj[f] ∧ Surj[f] (4.1.66)
∀x f(f(x)) = x (4.1.67)
Inv[f] Bij[f] (4.1.68)
∀x, y f(x) = f(y) (4.1.69)
x = y (4.1.70)
Sciences.ch 75/3049

Draft
∀x f(f(x)) = x, ∀y f(f(y)) = y (4.1.71)
f(x) = f(y) (4.1.72)
f(f(x)) = f(f(y)) (4.1.73)
x = y (4.1.74)
∀y∃x f(x) = y (4.1.75)
x = f(y) (4.1.76)
f(f(y)) = y (4.1.77)
Inj[f] ∧ Surj[f] (4.1.78)
(∀i)∀x, y f(x) = f(y) (4.1.79)
(→i)x = y (4.1.80)
f(x) = f(y) (4.1.81)
76/3049 Sciences.ch

Draft
f(f(x)) = f(f(y)) (4.1.82)
x = y (4.1.83)
∀y∃xf(x) = y(∀i) (4.1.84)
(∃i)x = f(y) (4.1.85)
f(f(y)) = y (4.1.86)
(∀e, ax) (4.1.87)
Inv[f] ∀y(f(f(y)) = y)
ax
Inv[f] {f(y) = x}[x := f(y)]
∀e
f(x) = f(y) f(x) = f(y)
ax
f(x) = f(y) {f(x) = y}[x := f(y)]
=e (4.1.88)
Inv[f] ∀x(f(f(x)) = x)
ax
Inv[f] {f(x) = y}[x := f(y)]
∀e
f(x) = f(y) f(x) = f(y)
ax
f(x) = f(y) {f(y) = x}[y := f(x)]
=e (4.1.89)
(??)(??)
f(x) = f(y) f(f(x)) = f(f(y))
=e (4.1.90)
Inv[f] → Inv[f]
ax
Inv[f] f(f(x)) = x
∀e
Inv[f] → Inv[f]
ax
Inv[f] f(f(y)) = y
∀e
f(f(x)) = f(f(y)) x = y
=e (??)
Inv[f], f(x) = f(y) x = y
Inv[f] f(x) = f(y) → x = y
Inv[f] Inj[f]
∀i
→i
=e (4.1.91)
Sciences.ch 77/3049

Draft
(5)(6)
Inv[f] Inj[f] ∧ Surj[f]
∧i (4.1.92)
Inv[f] Bij[f] ∀i (4.1.93a)
1 := Inv[f] Inj[f]
2 := Inv[f] Surj[f]
(1)Inv[f] Inj[f] ∀i (4.1.93b)
Inv[f] f(x) = f(y) → x = y →i
Inv[f], f(x) = f(y) x = y =e ×2(i)(ii)(iii)
(i) Inv[f] f(f(x)) = x ∀e
Inv[f] Inv[f] ax
(ii) Inv[f] f(f(y)) = y ∀e
Inv[f] Inv[f] ax
(iii) f(x) = f(y) f(f(x)) = f(f(x)) =c
(1 )f(x) = f(y = f(x) = f(y) ax
(2)Inv[f] Surj[f] ∀i (4.1.93c)
Inv[f] ∃x{f(x) = y} ∃i
Inv[f] {f(x) = y}[x := f(y)] ∀e
Inv[f] ∀x{f(f(x)) = x} ax
∼80 %
64 votes, 80.94%
Version: 3.1 Revision 5 | Last update: 2015-09-06 16:33
78/3049 Sciences.ch

Draft
4.2 Numbers
T
He basis of mathematics, apart the reasoning (see section Proof Theory), is undoubt-
edly to ordinary people: arithmetic. It is therefore mandatory that we make a stop
on it to study its origin, some of its properties and consequences.
The numbers, like geometric figures are the basis of Geometry, are the basis of Arithmetic.
These are also the historical basis because mathematics probably started with the study of these
objects, but also the educational foundation, because it is by learning to count that we enter in
the world of mathematics.
The history of numbers, also sometimes called scalar is far too long to be told here, but we
can only advise you one of the best book on the subject: The Universal History of Numbers
( 2,000 pages in three volumes) Georges Ifrah, ISBN: 2221057791.
But here’s a little flange of this latter which seems fundamental to us:
Our current decimal system, on base 10, uses the digits 0 to 9, called Arabic numbers, but the
fact of Indian origin (Hindus). The first numbers seems to have been created in the third century
BC in India by Brahmagupta, an Indian mathematician. he created the figures in Devanagari.
Indeed, the Arabic numbers (of Indian origin...) in the table below are the first line and we see
that they are significantly different from the Indian numbers of the second line:
Figure 4.2 – Indo-Arabic numbers
You have to read this table as following from left to right: 0 zero, 1 one, 2 two, 3 three,
4 four, 5 five, 6 six, 7 seven, 8 eight, 9 nine. This system is much more efficient than
the Roman numerals (try doing a calculation with Roman notation system you will see...).
It is commonly accepted that these numbers were introduced in Europe only about the year
1,000. Used in India, they were transmitted by Arabs to the Western world by the Pope Gerbert
of Aurillac during his stay in Andalusia at the end of the 9th century.
Remark
The French word chiffre (number) is a corruption of the Arabic word sifr meaning
zero. In Italian, zero is zero, and seems to be a contraction of zefiro, we again see
here an Arabic root but the zero could also be of Indian origin... So the words chiffre
and zero have the same origin.
The early use of a numerical symbol for the nothing in the sense of no amount, i.e. our
Sciences.ch 79/3049

Draft
zero is because the Indians used a system called positional system. In such a system, the
position of a digit in the writing of a number expresses the power of 10 and the number of times
it occurs ... and the absence of a position in this system arise from huge proofreading problems
and could lead to large errors in calculations. The revolutionary and simple introduction of the
concept of nothing allowed a proofreading without error of numbers.
The absence of a power is denoted by a small circle...: the zero. Our current system is thus the
decimal and positional system.
Example:
Figure 4.3 – Description of decimal and positional system
The number 324 is written from left to right as three hundred: 3 times 100, two tens: 2
times 10 and four units: 4 times 1.
Thus a decimal number is thus a number that has a ﬁnite writing in base 10.
We sometimes see (and this is recommended) a thousands separator represented by a coma in
United States (put all three numbers from the ﬁrst from the right for the whole numbers). Thus,
we write 1,034 instead of 1034 or 1,344,567,569 instead of 1344567569. Thousand separators
permits to quickly quantify the magnitude of the read numbers.
So:
• If we see only one coma we know that the number is about thousands
• If we see two apostrophes we know that the number is about millions
• If we see three apostrophes we know that the number is about billions
• etc.
and so on... also with decimals this gives:
80/3049 Sciences.ch

Draft
Figure 4.4 – Scale representation of the positional system
In fact, any integer other than the unit can be taken as the basis of a numbering system. We have
for example the binary, ternary, quaternary, ..., decimal, duodecimal numbering systems which
correspond respectively to the bases two, three, four, ..., ten, twelve.
A generalization of what has been seen above, can be written as follows:
Any positive integer N can be represented in a base b as a sum, where each coefﬁcient ai are
multiplied by their respective weight bi
. Such as:
N = an−1bn−1
+ an−2bn−2
+ ... + a1b1
+ a0b0
(4.2.1)
More elegantly written:
N =
n−1
i=0
aibi
(4.2.2)
with ai ∈ [0, b − 1] and bi ∈ [1, bn−1
]
Remarks
R1. As frequently in mathematics, we will replace numbers with Latin or Greek letters
in order to generalize their representation. So when we speak of a base b, the value of b
can take any positive integer value 1, 2, 3,...
R2. When we take the value 2 for b, the maximum value of N will be 2n
− 1.
The numbers that are written in this form are called Mersenne numbers. These
numbers can be prime numbers (see further below what a prime number is) if and only
if n is also a prime number.
Indeed, if we take (for example) b = 10 and n = 3 the largest value we can get
will be:
(b − 1)102
+ (b − 1)101
+ (b − 1)100
= 900 + 90 + 9 = 999 = 103
− 1 = b3
− 1
(4.2.3)
R3. When a number is the same read from left to right or right to left, we name it a
palindrome.
Sciences.ch 81/3049

Draft
4.2.1 Digital Bases
To write a number in base b system, we must first adopt b characters for represent-
ing the b first numbers for example in the decimal system: {0, 1, 2, 3, 4, 5, 6, 7, 8, 9}.
These characters are as we already defined them, the digits that we pronounce as usual
{zero, one, two, three, four, five, six, seven, eigth, nine}.
For the written numbers, we make this convention that a digit, placed to the left of another
represents the order units immediately above, or b times larger. To take the place of units that
may be lacking in certain orders, we use the zero 0 and consequently, the number of digits
may vary.
Definition: For the spoken numbers, we agree to name single unit, ten, hundred, thou-
sand, etc., units of the first, second, third, fourth order, etc. Thus the numbers 10, 11, ..., 19
will be readen in the same way in all numbering systems. The numbers 1a, 1b, a0, b0, ... will be
readen ten-a, ten-b, a-ten, b-ten, etc. Thus, the number 5b6a71c will be readen:
five million be-hundred sixty-a thousand seven hundred ten-c
This small example is relevant because it shows the general expression of the spoken language
we use daily is intuitively in base ten (fault of our education).
Remarks
R1. The rules of mathematical operations defined for numbers written in the decimal
system are the same for numbers written in any numbering system.
R2. To quickly operate in any numbering system, it is useful to know by heart all sums
and products of two numbers of a single digit.
R3. The decimal seems has its origin in the fact that humans being have ten fingers.
Let’s see how we convert a numbering system in another one:
Examples:
E1. In base ten we have seen above that 142, 713 will be written as:
142, 71310 = 1 · 105
+ 4 · 104
+ 2 · 103
+ 7 · 102
+ 1 · 101
+ 3 · 100
(4.2.4)
E2. The number 0110 that is in base two (binary base) would be written in base 10:
01102 = 0 · 23
+ 1 · 22
+ 1 · 21
+ 0 · 20
= 610 (4.2.5)
and so on... The reverse operation is often a little trickier (for example the case of the binary
base):
82/3049 Sciences.ch

Draft
Examples:
E1. Converting the decimal number 1, 492 in binary base is done by successive
divisions by 2 of residues and gives (the principle is about the same for all other bases):
Figure 4.5 – Decimal to binary conversion
E2. To convert the number 142, 713 (decimal base) in duodecimal base (base twelve) we
have (notation: q is the quotient, and r is the residue):
142, 713
12
⇒ q = 11, 892 r = 9
11, 892
12
⇒ q = 991 r = 0
991
12
⇒ q = 82 r = 7
82
12
⇒ q = 6 r = 10
6
12
⇒ q = 0 r = 6
(4.2.6)
Thus we have the residues 6, 10, 7, 0, 9 which leads us to write:
142, 71310 = 6a70912 (4.2.7)
where we have chosen for this particular example the symbolism that we have previously
deﬁned (a-ten) to avoid any confusion.
Sciences.ch 83/3049

Draft
4.2.2 Type of Numbers
Now that we know that number is a mathematical object used to count, measure and label it
must be know that it exists in mathematics a wide variety of numbers (natural, rational, real,
irrational, complex, p-adic quaternions, transfinite, algebraic, constructibles, etc.) since any
mathematician may at leisure create its own numbers just by defining axioms (rules) for the
manipulating them (see section Set Theory).
However, there are a few of them that we find much more often than others through this book
and some that serve as basic construction for others and which that should be defined sufficiently
rigorously (without going to the extremes) in order to know what we will talk about when we
will use them.
4.2.2.1 Natural Integer Numbers
The idea of integer (the numbers for which there are no decimals) is the fundamental concept
of mathematics and comes at the view of a group of objects of the same types (a sheep, another
sheep, yet another sheep, etc.).
When the amount of objects in a group is different from that of another group when the speak
about a group that is numerically higher or lower regardless of the type of objects in these
groups. When the amount of objects of one or multiples groups is equivalent, then we speak
about equality.
To each single object the number one or unit denoted by 1 in the decimal system will be
used.
To form groups of objects, we can operate as follows: to an object, add another object, then
another, and so on... each of of the clusters, from the point of view of its community, is charac-
terized by a number. It follows from this that a number can be regarded as representing a group
of units (single items) such that each unit corresponds to one single object of the collection.
Definition: Two numbers are said to be equal if each of the units of one we can match a
unique unit of the other and vice versa (in a bijective way as seen in the section of Set Theory).
If this does not hold true when we talk about inequality.
Let us take an object, then another, then to the formed group add again an object and so on. The
groups thus formed are characterized by numbers which, taken in the same order as the groups
successively obtained, are the natural sequence N, also sometimes named whole numbers,
and denoted by:
N = {0, 1, 2, 3, ...} (4.2.8)
To be unambiguous about whether 0 is included or not, sometimes an index (or superscript) is
added in the former case:
N∗
= N1
= {1, 2, 3, ...} (4.2.9)
84/3049 Sciences.ch

Draft
Remark
The presence of the 0 (zero) in our definition of N is debatable since it is neither positive
nor negative. That is why in some books you will find a definition of N without the 0.
The components of this natural set can be defined by (we own this definition to the mathemati-
cian Frege Gottlob) the following the properties (having read first the section on Set Theory is
strongly recommended...):
P1. 0 (read zero) is the number of elements (defined as an equivalence relation) of all sets
equivalent to (in bijection with) the empty set.
P2. 1 (read one) is the number of elements of all sets equivalent to the set whose only
element is 1.
P3. 2 (read two) is the number of elements of all sets equivalent to the set whose only
element are 1 and 2.
P4. In general, an integer is the number of elements of all sets equivalent to the set of integers
preceding it!
The construction of the set of natural numbers is made of the most natural and consistent man-
ner. Natural numbers get their name from what they were, in the beginnings of their existence,
to count quantities and things of nature or intervened in human life. The originality of this set
lies in the empirical way he has been built since it does not actually the result of a mathematical
definition, but more by awareness of the human by the concept of countable quantity, of number
and operations that reflect the relations between them.
The question about the origin of N is therefore the question of the origin of mathematics. And
since thousands of years debates confronting the thoughts of the greatest philosophical minds
have attempted to elucidate this deep mystery as to whether mathematics is a pure creation of
the human mind or whether the man has only rediscovered a science that already existed in
nature. Besides the many philosophical questions that the set of Natural numbers can generate,
it is nonetheless interesting from an exclusively mathematical point of view. Because of its
structure, it has remarkable properties that can be very useful when we practice some given
reasoning or calculations.
The sequence of natural numbers is unlimited (see section Theory Of Numbers) but countable
(we will this property in details below), because in a group of objects that is represented by a
number n, it will be enough to add an object to get another group that will be defined by the
integer n + 1.
Definition: Two integers that differs from a single positive unit are said to be consecutive.
Sciences.ch 85/3049

Draft
4.2.2.1.1 Peano axioms
During the crisis of foundations of mathematics, mathematicians have obviously sought
to axiomatize the set N and we own the actual axiomatisation to Peano and Dedekind.
The axioms of this system include the symbols and = to represent the relations smaller than
and equal to (see section Operators). They include also the symbols 0 for the number zero
and s to represent the successor number. In this system, 1 is denoted by:
1 = s(0) (4.2.10)
named successor to zero and 2 is denoted by:
2 = s(s(0)) = s(1) (4.2.11)
The Peano axioms that builds N are (see section of Proof Theory for details on some of the
symbols use below):
A1. 0 is a natural number (this permits to be not empty).
A2. Every natural number n has a successor, denoted by s(n). Therefore s is an injective
application (see section Set Theory), that is to say:
∀x, y s(x) = s(y) ⇔ x = y (4.2.12)
That is to say that if two successors are equal, they are the successors of the same number.
A3. The successor of a natural number is never zero (therefore N has a ﬁrst element):
∀x ¬(s(x) = 0) (4.2.13)
A4. If we prove a property ϕ that is true for x and its successor s(x), then this property is true
for any x (axiom of recurrence):
(ϕ(x) ⇒ ϕ(s(x))) ⇒ ∀x ϕ(x) (4.2.14)
So the set of all the numbers satisfying the four above axioms is denoted by:
So the set of all the numbers satisfying the four above axioms is denoted by:
N = {0, 1, 2, 3, ..., n, ...} (4.2.15)
Remark
The Peano axioms allow to build very rigorously the two basic operations of arithmetic
in N that are addition and multiplication (see section on Operators) and so all the other
sets that we will see later (subtraction in N can not be applied because it can give negative
numbers).
86/3049 Sciences.ch

Draft
4.2.2.1.2 Odd, Even and Perfect Numbers
In arithmetic, study the parity of an integer, its determiner if this integer is or not a
multiple of 2. An integer multiple of 2 is an even integer, the others are odd integers.
Deﬁnitions:
D1. The numbers obtained by counting by step of 2 from zero (i.e.. 0, 2, 4, 6, 8, ...) in the
set of natural integer numbers N are named even numbers. The nth
even number is
obviously given by the relation:
2n = n + n (4.2.16)
D2. The numbers we get by counting by step of 2 starting from 1 (i.e.. 1, 3, 5, 7, ...) in the
set of natural integer numbers N are named odd numbers. The (n + 1)th
even number
is almost obviously given by the relation:
2n + 1 (4.2.17)
Remarks
We name perfect numbers, numbers equal to the sum of their integer divisors strictly
smaller than themselves (concept we will see in detail later) such as: 6 = 1 + 2 + 3 and
28 + 1 = 2 + 4 + 7 + 14.
4.2.2.1.3 Prime Numbers
Deﬁnition: A prime number is an integer with exactly two positive divisors (these di-
visors are both: 1 and the number itself). In the case where there are more than two dividers
it named a composite number. The property of being prime (or not) is named primality.
The study of prime numbers is a huge subject in mathematics (see for a small example the
section of Number Theory or of Cryptography). There are books of thousands of pages on the
subject and probable hundreds of research article per month even nowadays. Most theorems are
largely out of the study of the site book (and out of the interest of its main author...)!
Here is the set of prime numbers less than 1000:
2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97, 101,
103, 107, 109, 113, 127, 131, 137, 139, 149, 151, 157, 163, 167, 173, 179, 181, 191, 193, 197,
199, 211, 223, 227, 229, 233, 239, 241, 251, 257, 263, 269, 271, 277, 281, 283, 293, 307, 311,
313, 317, 331, 337, 347, 349, 353, 359, 367, 373, 379, 383, 389, 397, 401, 409, 419, 421, 431,
433, 439, 443, 449, 457, 461, 463, 467, 479, 487, 491, 499, 503, 509, 521, 523, 541, 547, 557,
563, 569, 571, 577, 587, 593, 599, 601, 607, 613, 617, 619, 631, 641, 643, 647, 653, 659, 661,
673, 677, 683, 691, 701, 709, 719, 727, 733, 739, 743, 751, 757, 761, 769, 773, 787, 797, 809,
811, 821, 823, 827, 829, 839, 853, 857, 859, 863, 877, 881, 883, 887, 907, 911, 919, 929, 937,
941, 947, 953, 967, 971, 977, 983, 991, 997
Sciences.ch 87/3049

Draft
The whole set of prime numbers is sometimes denoted by P.
Remark
Note that the primes numbers set does not include the number 1 because it has a only a
single divider (himself) and not two as is the definition.
We can ask ourselves if there are infinitely many prime numbers? The answer is YES and here
is a proof (among others) by contradiction.
Proof 4.0.1. Suppose that there is a finite number of prime numbers that would be denoted by:
p1, p2, ..., pn (4.2.18)
We create a new number from the product of this prime number to which we add 1:
N = (p1p2...pn) + 1 (4.2.19)
According to our initial hypothesis and the fundamental theorem of arithmetic (see section
Number Theory) the new number N should be divisible by one of the existing prime pi such
that we can write:
N = q · pi (4.2.20)
where q is an integer. We can make the division:
q =
(p1p2...pn) + 1
pi
=
p1p2...pn
pi
+
1
pi
(4.2.21)
The first term is simplified as pi is in the product. Let us note the resulting integer E:
q = E +
1
pi
(4.2.22)
But, q and E are integers, so 1/pi should be an integer. But pi is by definition greater than 1.
So 1/pi is not an integer and so is also q.
Then there is contradiction, and we can conclude that the prime numbers are not finite but are
infinite.
Q.E.D.
Remarks
R1. The product pn = p1p2...pn of the indexed prime numbers ≤ n is named the n-th
primorial.
R2. We send the reader to the section Cryptography of the chapter on Theoretical Com-
puting (or Number Theory section of the chapter Arithmetic) for the study of some re-
markable properties of prime numbers including the famous Euler φ function (also named
indicator function) and a 20th-21th century industrial application of prime numbers.
88/3049 Sciences.ch

Draft
4.2.2.2 Relative Integer Numbers
The set of natural integers N has a few issues that we did not set out earlier. For example, sub-
tracting two numbers into N does not always have a result in N (negative numbers not existing
in this set). Other issue... dividing two numbers in N also does not always have a result in N
(fractional numbers - rational or irrational - not existing in this set). We then say in the language
of set theory that: the substraction and division is not an internal operation of N.
We can first resolve the problem of subtraction by adding to the set of natural numbers N,
negative integers (revolutionary concept for those who where behind this concept at their time!)
to get the set of relative integers denoted by Z (for Zahl from German, meaning Number):
Z = ..., −3, −2, −1, 0, 1, 2, 3, ... (4.2.23)
The set of natural integers is therefore included in the set of relative integers. This is what we
denote by (see section Set Theory):
N ⊂ Z (4.2.24)
and we have by definition (it is a notations to be learned!!!):
Z+
= Z0 = {n ∈ Z|n 0} = N∗
Z+
0 = Z≥0 = {n ∈ Z|n ≥ 0} = N
Z−
0 = Z0 = {n ∈ Z|n 0}
Z∗
= Z=0 = {n ∈ Z|n = 0}
(4.2.25)
This set was originally created to make the natural numbers an object that we name a group
(see section Set Theory) relatively to the addition.
Definition: We say that a set A is a countable set, if it is equipotent to N. That is to say if
there is a bijection (see section Set Theory) of S on N. Thus, roughly said, two equipotent sets
have the same number of elements in the meaning of their cardinal (see section Set Theory), or
at least the same infinity.
The purpose of this concept is to understand that the sets N and Z are countable.
Proof 4.0.2. Let us show that Z is countable by writing:
x2k = k and x2k+1 = −k − 1 (4.2.26)
for any integer k ≥ 0. This gives the following ordered list:
0, −1, 1, −2, 2, −3, 3, ... (4.2.27)
of all relative integers from natural integers only!
Q.E.D.
Sciences.ch 89/3049

Draft
4.2.2.3 Rational Numbers
The set of relative integers Z also still has an issue. Dividing two numbers in Z also does not
always have a result in Z (fraction numbers - rationnal or irrational - not existing in this set).
We then say in the language of set theory that: the division is not an internal operation of Z.
We can thus define a new set that contains all the numbers which can be written as a fraction
that is to say the ratio of a dividend (numerator) and a divider (denominator). When a number
can be written in this form, we say that it is a fractional number:
fraction :
2
3
Numerator
←− fraction bar
Denominator
(4.2.28)
A fraction can be used to express a part or fraction of something (of an object, of a distance, of
a land, of an amount of money, of a cake...).
By definition, the set of rational numbers is given by:
Q =
p
q
|(p, q) ∈ Z, q = 0 (4.2.29)
In other words, any rational number is any number that can be expressed as the quotient or
fraction p/q of two integers, p and q, with the denominator q not equal to zero. Since q may be
equal to 1, every integer is a rational number.
We also assume as obvious that:
N ⊂ Z ⊂ Q (4.2.30)
The logic of the creation of the set of rational numbers Q is similar to that of relative integers
Z. Indeed, mathematicians wanted to make of the set relative numbers Z a group with respect
to the law of multiplication and division (see section Set Theory).
Moreover, contrary to the intuition of most people, the set of natural integers N and rational
numbers Q are equipotent. We can convince ourselves of this equipotence by ranking ,as Cantor
did, rational numbers in a first time as follows:
90/3049 Sciences.ch

Draft
1/1 2/1 3/1 4/1 5/1 6/1 7/1 · · ·
1/2 2/2 3/2 4/2 5/2 6/2 7/2 · · ·
1/3 2/3 3/3 4/3 5/3 6/3 7/3 · · ·
1/4 2/4 3/4 4/4 5/4 6/4 7/4 · · ·
1/5 2/5 3/5 4/5 5/5 6/5 7/5 · · ·
1/6 2/6 3/6 4/6 5/6 6/6 7/6 · · ·
1/7 2/7 3/7 4/7 5/7 6/7 7/7 · · ·
...
...
...
...
...
...
... · · ·
Figure 4.6 – Cantor diagonal method
This table is constructed so that each rational number appears only once (in the sense of its
decimal value) by diagonal hence the name of the method: Cantor diagonal.
If we eliminate from each diagonal the rational numbers that appear more than one time (the
equivalent fractions) in order to keep only those who are irreducible (i.e. those with the
greatest common divisor of the numerator and denominator is equal to 1), then we can with this
distinction define an application f : N ⇒ Q that is injective (two distinct rational numbers have
distinct ranks) and surjective (at any place will be written a rational number).
The application f is therefore bijective: N and Q are then effectively equipotent!
The definition a little bit more rigorous (and therefore less funny) of Q from Z is as follows (it
is interesting to see the notation used):
On the set Z × Z {0}, which should be read as the set constructed from two relative integer
whose zero is excluded from the second one, we consider the relation R between two relative
pairs of integers defined by:
(a, b)R(a , b ) ⇔ ab = a b (4.2.31)
We then easily verify that R is an equivalence relation (see section Operators) on Z × Z {0}.
The set of equivalence classes for this relation R denoted then (Z × Z {0}) /R is by definition
Q. That is to say that we write therefore more rigorously:
Q = (Z × Z {0})/R (4.2.32)
The equivalence class (a, b) ∈ Z × Z {0} is explicitly denoted by:
a
b
(4.2.33)
Sciences.ch 91/3049

Draft
in accordance with the notation that everyone is accustomed to use.
We easily check the addition and multiplication operations that were operations defined on Z
pass without problems to Q by writing:
a
b
·
p
q
=
a · p
b · q
and
a
b
+
p
q
=
aq + bp
bq
(4.2.34)
Moreover, these operations provide Q with the structure of a body (see section Set Theory) with
0
1
as neutral element for the addition and
1
1
as neutral element for the multiplication. Thus, any
non-zero element of Q is reversible, in fact:
a
b
·
b
a
=
ab
ab
=
1
1
(4.2.35)
what is written also more technically:
(ab, ab)R(1, 1) (4.2.36)
Remark
Even if we want to define Q as the being the set Z × Z 0 where Z represents the
numerators and Z 0 the denominators of the rationals, this is not possible because
otherwise we would for example (1, 2) = (2, 4) while we expect for an equality.
Hence the need to introduce an equivalence relation which enables us to identify, to return
to the previous example, with (1, 2) and (2, 4). The relation R that we have defined does
not fall from heaven, indeed the reader who handled the rational so far without ever
having seen their formal definition knows that:
a
b
=
a
b
⇔ ab = a b (4.2.37)
It is therefore almost natural to define the relation R as we have done. In particular,
regarding the above example,
1
2
=
2
4
because (1, 2)R(2, 4) and the problem is solved.
In addition to the historical circumstances of its establishment, this new entity (set) is distin-
guished from relative numbers because it induces the original and paradoxical concept of par-
tial quantities. This notion that a priori does not make sense, find its place in the mind of man
thanks to the geometry where the idea of fraction of length, of proportion are illustrated more
intuitively.
92/3049 Sciences.ch

Draft
4.2.2.4 Irrational Numbers
It can seem obvious to present irrational numbers before real number (see further below) but
this can be explained by the fact of this is the order of the discovering in the human history and
therefore is seems more pedagogical to us to present them in this order.
So the set of rational Q is limited and sadly not sufficient too. Indeed, we may think that all
mathematical computation with commonly known operations are reduced to this set but it is not
the case!
Examples:
E1. Let us calculate the square root of two which we denote
√
2 (thing to Pythagorean
theorem with a triangle of side 1 and 1 then the third one is of size
√
2). Suppose it is a
rational root. So if this is truly a rational, we should be able to express it as a/b, where
by the definition of a rational a and b are integers with no common factors. For this
reason, a and b can not both be even numbers. There are three remaining possibilities:
1. a is odd (then b is even)
2. a is even (then b is odd)
3. a is odd (then b is odd)
By squaring, we have:
√
2 =
a
b
(4.2.38)
That can be written:
2b2
= a2
(4.2.39)
Since the square of an odd number is odd and the square of an even number is even, the
case (1) is not possible because a2
would be odd and 2b2
would be even.
If case (2) is also impossible, because then we could write a = 2c, where c is any integer,
and so if we take the square then we have a2
= 4c2
that is to say an even number on both
sides of equality. Substituting in 2b2
= a2
we obtain after simplification that b2
= 2c2
.
Then b2
would be odd while 2c2
would even.
The case (3) is also impossible because a2
is then odd and 2b2
is even (that b is even or
odd!).
There is the no solutions! That is to say that the start assumption is false and there does
not two integers a and b such that
√
2 = a/b.
Sciences.ch 93/3049

Draft
E2. Let us prove by contradiction, that the famous Euler number e is irrational. To do
this, remember that e (see section Functional Analysis) can also be defined by the Taylor
series (see section Sequences and Series):
e = 1 +
1
1!
+
1
2!
+
1
3!
+ . . . +
1
n!
+ . . . (4.2.40)
Then if e is rational, it could be written in the form p/q (with q 1, because we know
that is not an integer). Let us multiply both sides of the equality by q!:
q!e = q! +
q!
1!
+
q!
2!
+
q!
3!
+ . . . +
q!
q!
+
q!
(q + 1)!
+
q!
(q + 2)!
+ . . . (4.2.41)
The first member q!e would then be an integer, because by definition of the factorial:
q! = q · (q − 1) · (q − 2) . . . 2 · 1 (4.2.42)
is an integer.
The first terms of the second member of the previous prior-previous relation, until the
term q!/q! = 1 are also integer because q!/m! is simplified if q m. So by subtraction
we find:
q!e − q! +
q!
1!
+
q!
2!
+
q!
3!
+ . . . +
q!
q!
+ . . . =
q!
(q + 1)!
+
q!
(q + 2)!
+ . . . (4.2.43)
when the right sequences should be an integer!
After simplification, the second member of the equality becomes:
1
q + 1
+
1
(q + 1)(q + 2)
+ . . . (4.2.44)
the first term in this sum is strictly less than 1/2, the second strictly less than 1/4 second,
the third strictly less than 1/8, etc.
So, since each term is strictly less than the following harmonic series which converges to
1:
1
2
+
1
4
+
1
8
+ . . . = 1 (4.2.45)
then therefore the sequence is not an integer as being strictly less than 1. This is a con-
tradiction!
Thus, the rational numbers do not satisfy the numerical expression of
√
2 and e (to cite only
these two particular examples).
They must therefore be complemented by the set of all numbers that can not be written as a
fraction (the ratio of an integer dividend and an integer divisor without common factors) and
that we name irrational numbers. Finally we can say that:
94/3049 Sciences.ch

Draft
Definition: In mathematics, an irrational number is any real number that cannot be expressed
as a ratio of integers. Irrational numbers cannot be represented as terminating or repeating
decimals.
4.2.2.5 Real Numbers
Definition: The union of rational and irrational numbers gives the set of real numbers that we
denote by:
Q ⊂ R (4.2.46)
Remark
Mathematicians in their usual rigor have different techniques to define real numbers.
They use the properties of topology (among others) and especially Cauchy sequences but
that’s another story that goes beyond the formal scope of this section. For a set point of
view definition of R the reader should report to the section on Set Theory.
Figure 4.7 – Simple number sets summary
Obviously we are led to ask ourselves whether R is countable or not. The proof is quite simple.
Proof 4.0.3. By definition, we have seen above that there must be a bijective correspondence
between Q and R to that R is countable.
For simplicity, we will show that the interval [0, 1[ is then not countable. This will involve of
course by extension that R is not countable!
The elements of this interval are represented by infinite sequences between 0 and 9 (in the
decimal system):
• Some of these suites are zero from starting from a given rank, some not.
• So we can identify [0, 1[ to the set of all sequences (finite or infinite) of integers between
0 and 9.
Sciences.ch 95/3049

Draft
n◦
1 x11 x12 x13 x14 · · · x1p · · ·
n◦
2 x21 x22 x23 x24 · · · x2p · · ·
n◦
3 x31 x32 x33 x34 · · · x3p · · ·
n◦
4 x41 x42 x43 x44 · · · x4p · · ·
n◦
5 x51 x52 x53 x54 · · · x5p · · ·
n◦
6 x61 x62 x63 x64 · · · x6p · · ·
· · ·
· · ·
· · ·
n◦
k xk1 xk2 xk3 xk4 · · · xkp · · ·
· · ·
If this set was countable, we could classify these sequences (with a first, second, etc.).
Thus, the sequence x11x12x13x14...x1p... would be classified first and so on ... as proposed
in the above table.
We could then edit this infinite matrix as follows: to each element of the diagonal, we add
1, according to the rule: 0 + 1 = 1, 1 + 1 = 2, 8 + 1 = 9 and 9 + 1 = 0:
n◦
1 x11 + 1 x12 x13 x14 · · · x1p · · ·
n◦
2 x21 x22 + 1 x23 x24 · · · x2p · · ·
n◦
3 x31 x32 x33 + 1 x34 · · · x3p · · ·
n◦
4 x41 x42 x43 x44 + 1 · · · x4p · · ·
n◦
5 x51 x52 x53 x54 · · · x5p · · ·
n◦
6 x61 x62 x63 x64 · · · x6p · · ·
· · ·
· · ·
· · ·
n◦
k xk1 xk2 xk3 xk4 · · · xkp · · ·
· · ·
Then let us consider the sequence on the diagonal:
– It cannot be equal to the first sequence of the first row in the prior-previous table
since it is distinguished at least by the first element.
– It cannot be equal to the second sequence of the second row of the prior-previous
table since is distinguished at least by the second element.
– It cannot be equal to the third sequence of the second row of the prior-previous table
since is distinguished at least by the third element.
and so on ... It the cannot be equal to any of the sequences in this table!
So whatever the chosen classification of infinite sequences of 0...9, there is always one who
escapes this classification! So it is that it is impossible to number them ... simply because they
do not form a countable set!
Q.E.D.
96/3049 Sciences.ch

Draft
The technique that has allowed us to achieve this result is known as the Cantor diagonal pro-
cess (because similar to that used for equipotence between the natural and rational set) and the
set of real numbers is said to have the power of continuum by the fact that it is uncountable.
Remark
We assume that it is intuitive for the reader intuitive that any real number can be approx-
imated infinitely close by a rational number (for irrational numbers we simply stop at
a given number of decimals and find the corresponding rational). Mathematicians say
therefore that Q is dense in R and denote this by:
¯Q = R (4.2.47)
In business it is of usage with real numbers to communicate in percentages or per-thousand.
Definitions:
D1. Given a scalar x ∈ R then expressed in percentage it will denoted by:
x% = x · 100 (4.2.48)
D2. Given a scalar x ∈ R then expressed in per-thousand it will denoted by:
x‰ = x · 1, 000 (4.2.49)
4.2.2.6 Transfinite Numbers
We now are with an infinity of real numbers which is different from that of natural numbers.
Cantor then dared what no one had dared since Aristotle: the positive integers sequence is also
infinite, the set N, is then a set that has a countable infinity of elements, then he said that the
cardinal (see section Set Theory) of this set was a number that existed as such without we use
the tote symbol ∞, he denote it:
ℵ0 = Card(N) (4.2.50)
This symbol is as we know (see section Set Theory) the first letter of the Hebrew alphabet,
pronounced aleph zero. Cantor was going to name this strange number, a transfinite number.
The decisive act is to assert that there is, after the finite, a transfinite, that is to say an unlimited
scale of determined modes which by nature are infinite, and yet can be specified, as for the
finite, by specific numbers, well defined and distinguishable from each other!! This tool was
necessary as a set cardinal can be equal to one of its parts as we will see just below!
After this first stroke going against most ideas for over two thousand years, Cantor would con-
tinue its path and build the calculation rules, paradoxical at first glance, of the transfinite num-
bers. These rules were based, as we said earlier, on the fact that two infinite sets are equivalent
if there exists a bijection between the two sets.
Sciences.ch 97/3049

Draft
Thus, we can easily show that the infinity of even numbers is equivalent to the infinity of inte-
gers: for this, it suffices to show that for every integer, we can associate an even number, his
double, and vice versa. Therefore the cardinal of integers is equal to those of even numbers (the
cardinal of a set can be equal to one of its parts!).
Thus, although if even numbers are included in the set of integers, there is an infinity α0 of
them, the two sets are equipotent. By stating that a set can be equal to one of its parts, Cantor
goes against what seemed obvious to Aristotle and Euclid: the set of all sets is infinite! This
will shake the whole of mathematics and will bring the axiomatic Zermelo-Fraenkel we will see
in the section of Set Theory.
From the above, Cantor define the following calculations rules on the Cardinals:
ℵ0 + 1 = ℵ0 ℵ0 + ℵ0 = ℵ0 ℵ2
0 = ℵ0 (4.2.51)
At first glance these rules seem non-intuitive, but in fact they are! Indeed, Cantor defined the
addition of two transfinite numbers as cardinal of the disjoint union of the corresponding sets.
Examples:
E1. By noting ℵ0 the cardinal of N we have ℵ0 + ℵ0 which is equivalent to say-
ing that we summ the cardinal of N disjoint union N. But as N disjoint union N is
equipotent to N then ℵ0 + ℵ0 = ℵ0 (it is enough to be convinced to take the set of odd
and even integers which are both countable and which disjoint union is also countable).
E2. Other trivial example: ℵ0 + 1 corresponds to the cardinality of N union a point. This
set is still equipotent to N therefore ℵ0 + 1 = ℵ0.
We will also during our study of the section Set Theory that the concept of Cartesian product of
two countable sets is such that we have:
Card(N × N) = Card(N2
) = [Card(N)]2
(4.2.52)
and therefore:
ℵ2
0 = ℵ0 (4.2.53)
Similarly (see section Set Theory) since Z = Z+
∪ Z−
we have:
ℵ0 + ℵ0 = ℵ0 (4.2.54)
and identifying Q to Z × Z (ratio of a numerator over denominator) we have immediately:
ℵ0 × ℵ0 = (ℵ0)2
= ℵ0 (4.2.55)
We can also prove an interesting statement: if we consider the cardinality of the set of all the
cardinals, it is necessarily greater than all the cardinals, including itself (it is better to have read
previously the section of Set Theory)! In other words: the cardinality of the set of all sets of A
is greater than the cardinal of A itself.
98/3049 Sciences.ch

Draft
This implies that there is no set containing all sets since there is always a bigger one (it is an
equivalent form of the famous old Cantor’s paradox)!!!
In technical language it means considering a non-empty set A and then to state that:
Card(A) ≤ Card(P(A)) (4.2.56)
where P(A) is the set of subsets of A (see the section Set Theory for the general calculation of
the cardinal of the set of all parts of a countable set).
That is to say, by definition of the order relation (strictly less), it suffices to prove that there is
no surjective application f : A → P(A), in other words that to each element of the set of parts
of A it does not match at least one pre-image in A.
Remark
The set P(N) for example consists of the set of even numbers, odd numbers, natural
numbers, as well as the empty set itself, etc. P(N) is therefore the set of all potatoes
(to borrow the vocabulary of high school ...) that make N.
Proof 4.0.4. Suppose that we can number each potatoe of P(A) with at least one element of
A (imagine that with N or see the example in the section of Set Theory). In other words it is
equivalent to suppose that f : A → P(A) is surjective and let us consider a subset E of A such
that:
E = {x ∈ A|x /∈ f(x)} (4.2.57)
that is to say the set of elements x of A that do not belong to the set numbered by x (the element
x does not belong to the potato that it numbers in other terms...).
Or, if f is surjective it must also be a y ∈ A for this subset E such that:
f(y) = E = {x ∈ A|x /∈ f(x)} (4.2.58)
since E is also a subset of A.
Suppose that y belongs to E. In this case, by definition of E, y /∈ f(y) = E (by definition
of E that applies for every x and x can also be obviously y or z or don’t matter what). By
consequence, y inE, but in this second case, always by definition of E, y ∈ f(y) = E (as y is
not in E). We see therefore that the element y cannot exists and therefore f cannot be surjective.
We strongly recommend the reader to read the previous sentence more than on time if necessary.
Q.E.D.
Sciences.ch 99/3049

Draft
4.2.2.7 Complex Numbers
Invented in the 16th century among others by Girolamo Cardano and Rafaello Bombelli, com-
plex numbers (also named imaginay numbers) are used to solve problems with no solutions
in R and also used to mathematically formalize certain transformations in the plan such as ro-
tation, similarity, translation, etc. and also to generalized some theorem restricted to R and
therefore hiding some interesting results for practical engineering. For physicists, complex
numbers above are also a very convenient way to simplify notations. It is thus very difficult to
study wave phenomena, general relativity or quantum mechanics without using complex num-
bers and expressions.
There are several ways to construct complex numbers. The first is typical of the construction
way that mathematicians used as part of Set Theory. They define a couple of real numbers
and define the operations between these couples to finally arrive at a meaning of the complex
number concept. The second one is less rigorous but its approach is simpler and consist to
define the pure unit imaginary number i and then build arithmetic operations from its definition.
We will opt for the second method in the texts that will follow!
Definitions:
1. We define the unit pure imaginary number that we denote by i by the following property:
i2
= −1 ⇔ i =
√
−1 (4.2.59)
2. A complex number is a pair of a real number a and an imaginary number ib and gener-
ally written in the following form:
z = a + ib (4.2.60)
where a and b are numbers belonging to R.
3. We note the set of complex numbers by C and therefore we have by construction:
R ⊂ C (4.2.61)
Remark
The set C is identified to the oriented Euclidean plane E (see section Vector Calculus)
thanks to the choice of a direct orthonormal basis (we therefore get Argand-Cauchy
plane, also named Gauss-Argand plane or more commonly Gauss plane that we will
see a little further below and that seems have be defined for the first time in 1806).
The set of complex numbers that constitutes a field (see section Set Theory) and denoted by C,
is defined (in a simple way to start) in the notation of set theory by:
C = {z = (x + iy)|x, y ∈ R} (4.2.62)
100/3049 Sciences.ch

Draft
In other words we say that the field C is the field R to which we have added the imaginary
number i. Which is formally denoted by:
R [i] (4.2.63)
The addition and multiplication of complex numbers are internal operations to the set (field)
of complex numbers (we will come back much more in detail on certain properties of complex
numbers in the section of Set Theory) and defined by:
z1 + z2 = (x1 + x2) + i(y1 + y2)
z1 · z2 = (x1x2 − y1y2) + i(x1y2 + x2y1)
(4.2.64)
The real part of z is traditionally denoted by:
(z) = x (4.2.65)
The imaginary part of z is traditionally denotedby:
(z) = x (4.2.66)
The conjugate of z is defined by:
¯z = x − iy (4.2.67)
and is sometimes also denoted z∗
(particularly in quantum physics in some books!).
From a complex and its conjugate, it is possible to find its real and imaginary parts. These are
the following obvious relations:
(z) =
z + ¯z
2
and (z) =
z − ¯z
2i
(4.2.68)
The module of z (or norm) is the length from the center of the Gaussian plane (see further
below a figure of the Gaussian plane) and is simply calculated using the Pythagorean theorem :
|z| = x2 + y2 =
√
z · ¯z (4.2.69)
and is always a positive number or or equal to zero.
We consider as obvious that is satisfy all the properties of a distance (see section of Topology
and Vector Calculus).
Remark
The notation |z| for the module is not innocent since |z| coincides with the absolute value
of z when z is real.
The division between two complex number s is calculated as (the denominator is obviously not
zero):
z1
z2
=
x1 + iy1
x2 + iy2
=
x1 + iy1
x2 + iy2
·
x2 − iy2
x2 − iy2
=
(x1x2 + y1y2) − i(x1y2 − x2y1)
x2
2 + y2
2
(4.2.70)
The opposite of a complex number is calculated similarly:
1
x + iy
=
x − iy
(x + iy)(x − iy)
=
x − iy
x2 + y2
=
x
x2 + y2
− i
y
x2 + y2
(4.2.71)
We can therefore list 8 important properties of the module and the complex conjugate:
Sciences.ch 101/3049

Draft
P1. We afﬁrm that:
|z| = 0 ⇔ z = 0 (4.2.72)
Proof 4.0.5. By deﬁnition of the module |z| = x2 + y2 so that the sum x2
+ y2
is zero,
the necessary condition is that as (x, y) ∈ R:
x = y = 0 (4.2.73)
Q.E.D.
|z| = | − z| = |¯z| (4.2.74)
Proof 4.0.6. This is immediate by:
|z| = x2 + y2 = (−x)2 + (−y)2 = | − z| = (x)2 + (−y)2 = |¯z| (4.2.75)
Q.E.D.
| (z)| |z| with equality iif z is real
| (z)| |z| with equality iif z is imaginary
(4.2.76)
Proof 4.0.7. The two above inequalities can be written:
|x| ≤ x2 + y2
|y| ≤ x2 + y2
(4.2.77)
thus equivalent respectively to:
x2
≤ x2 + y2
y2
≤ x2 + y2
(4.2.78)
which are trivial. The rest of the proof is therefore trivial!
Q.E.D.
P4. We have:
∀z1, z2 ∈ C × C |z1z2| = |z1||z2| (4.2.79)
and if z2 = 0:
z1
z2
=
|z1|
|z2|
(4.2.80)

Draft
Proof 4.0.8. First:
|z1z2|2
= (z1z2)(z1z2) = (z1z2)(¯z1 ¯z2) = (z1 ¯z1)(z2 ¯z2) = |z1|2
|z2|2
⇒ |z1z2| = |z1||z2|
(4.2.81)
(we will prove a little further below that generally z1z2 = ¯z1 ¯z2) and for z2 = 0:
z1
z2
2
=
z1
z2
z1
z2
=
z1
z2
¯z1
1
z2
=
z1
z2
¯z1
1
¯z2
=
z1 ¯z1
z2 ¯z2
=
|z1|2
|z2|2
(4.2.82)
and taking root square this finish the proof.
Q.E.D.
|z|2
= z¯z (4.2.83)
Proof 4.0.9. This is immediate:
|z|2
= ( x2 + y2)2
= (x + iy)(x − iy) = x2
− ixy + iyx + y2
= x2
+ y2
= ¯zz
(4.2.84)
Q.E.D.
∀z, z ∈ C ¯¯z = z, z + z = ¯z + ¯z , zz = ¯z ¯z (4.2.85)
Proof 4.0.10. The first one is immediate:
¯¯z = x + iy = x − iy = x + iy = z (4.2.86)
and:
z + z = (x1 + iy1) + (x2 + iy2) = (x1 + x2) + i(y1 + y2) = (x1 + x2) − i(y1 + y2)
= (x1 − iy1) + (x2 − iy2) = ¯z + ¯z
(4.2.87)
and:
zz = (x1 + iy1)(x2 + iy2) = (x1x2 − y1y2) + i(x1y2 + y2x2)
= (x1x2 − y1y2) + i(x1y2 + y2x2) = (x1 − iy1)(x2 − iy2) = ¯z ¯z
(4.2.88)
Q.E.D.
Remark
R1. In mathematical terms, the first proof helps to show that complex conjugation
is what is named an involution (in the sense that it is changing anything ...).
R2. Also in mathematical terms (it is only the vocabulary!), the second proof
shows that the combination of the sum of two complex numbers is what we name
a group automorphism (C, +) (see section Set Theory).
R3. Again, for vocabulary ... the third proof show that the combination of the prod-
uct of two complex numbers is what we name a field automorphism (C, +, ×)
(see section Set Theory).

Draft
P7. We affirm that for z different from zero:
1
z
=
1
¯z
z
z
=
¯z
¯z
(4.2.89)
Proof 4.0.11. We will restrict ourselves to the proof of the second relation that is a general
case of the first (for z = 1).
z
z
=
x1 + iy1
x2 + iy2
=
x1 + iy1
x2 + iy2
x2 − iy2
x2 − iy2
=
(x1x2 + y1y2) + i(y1x2 − y2x1)
x2
2 + y2
2
=
x1x2 + y1y2
x2
2 + y2
2
− i
y1x2 − y2x1
x2
2 + y2
2
=
(x1x2 + y1y2) + i(y2x1 − y1x2)
x2
2 + y2
2
=
(x1 − iy1)(x2 + iy2)
x2
2 + y2
2
=
x1 − iy1
x2 − iy2
x2 + iy2
x2 + iy2
=
¯z
¯z
(4.2.90)
Q.E.D.
P8. We have:
|z1 + z2| |z1| + |z2| (4.2.91)
for any complex number z1, z2 (strictly speaking non-zero complex numbers, otherwise
the concept of argument of the complex number that we will see further below is undeter-
mined). Furthermore the equality holds if and only if z1 and z2 are collinear (the vectors
are on the same straight line) and of the same direction, in other words.... if it exist
λ ∈ R such as λz1 = z2.
Proof 4.0.12. Directly we have:
|z1 + z2|2
= |z1|2
+ 2 (z1 ¯z2) + |z2|2
|z1|2
+ 2|z1 ¯z2| + |z2|2
= (|z1| + |z2|)2
(4.2.92)
This inequality may not be obvious to everyone, therefore let us develop it a bit and let us
assume it true:
|z1 + z2|2
?
(|z1| + |z2|)2
(x1 + x2)2 + (y1 + y2)2
2 ?
x2
1 + y2
1 + x2
2 + y2
2
2
(x1 + x2)2
+ (y1 + y2)2
?
(x2
1 + y2
1) + 2 x2
1 + y2
1 x2
2 + y2
2 + (x2
2 + y2
2)
x2
1 + 2x1x2 + x2
2 + y2
1 + 2y1y2 + y2
2
?
(x2
1 + y2
1) + 2 x2
1 + y2
1 x2
2 + y2
2 + (x2
2 + y2
2)
(4.2.93)
After simplification:
x1x2 + y1y2
?
x2
1 + y2
1 x2
2 + y2
2
(x1x2 + y1y2)2
?
(x2
1 + y2
1)(x2
2 + y2
2)
x2
1x2
2 + 2x1x2y1y2 + y2
1y2
2
?
x2
1x2
2 + x2
1y2
2 + y2
1x2
2 + y2
1y2
2
(4.2.94)

Draft
and again after simplification:
2x1x2y1y2
?
x2
1y2
2 + y2
1x2
2
0
?
x2
1y2
2 − 2x1x2y1y2 + y2
1x2
2
0
?
(x1y2 − y1x2)2
(4.2.95)
So as the square brackets is necessarily positive or zero it follows:
0 ≤ (x1y2 − y1x2)2
(4.2.96)
This last relation thus shows that inequality is true.
Q.E.D.
Remark
In fact there is a more general form of this inequality named Minkowski inequal-
ity, proved in the section of Vector Calculus (complex numbers can indeed be
written in the form of vectors as we will see later).
4.2.2.7.1 Geometric Interpretation of Complex Numbers
We can also represent any complex number a + ib or a − ib in a plane defined by two
axes (two dimensions) of infinite length and orthogonal between them. The vertical axis
represents the imaginary part of a complex number and the horizontal axis the real part (see
figure below).
So there is correspondence between the set of complex numbers and the set of vectors of the
Gaussian plane (notion of affix as we will see more deeply in the section of Vector Calculus).
We sometimes named this type of representation Gauss plane or Gauss map:
Figure 4.8 – Complex Gauss plane
and then we write:
Aff(r) = a + ib (4.2.97)

Draft
We see on this diagram that a complex number has thus a vector interpretation (see section
Vector Calculus) given by:
z = a
1
0
+ b
0
1
= a + ib (4.2.98)
where the canonical basis is deﬁned such as:
1 =
1
0
, i =
0
1
(4.2.99)
with:
r = |z| =
√
a2 + b2 (4.2.100)
Thus,
1
0
is the unitary basis vector of the carried by the horizontal axis R and
0
1
is the
unitary basis vector carried by the vertical imaginary axis Ri and r is the module (the norm)
that is positive or zero.
This has to be compared with the vectors of R2
(see section Vector Calculus):
v = xe1 + ye2 = x
1
0
+ y
0
1
=
x
y
(4.2.101)
with:
||v|| = x2 + y2 (4.2.102)
so that we can identify the complex plane with the Euclidean plane. Thanks to the geometric
interpretation of the Gaussian plane, the equality below is immediate for example and avoids
making some developments:
a + bi
b + a
= 1 (4.2.103)
In addition, the deﬁnitions of the cosine and sine (see sectionTrigonometry) give us:
a = r cos(ϕ) b = r sin(ϕ) (4.2.104)
Finally:
r =
√
a2 + b2
ϕ−1
= cos−1 a
r
= sin−1 b
r
= tan−1 b
a
(4.2.105)
Therefore:
z = a + ib = r cos(ϕ(+ir sin(ϕ) = r(cos(ϕ) + i sin(ϕ)) = rcis(ϕ) (4.2.106)
complex number which is always equal to itself modulo 2π by the properties of trigonometric
functions:
z = r(cos(ϕ) + i sin(ϕ)) = r(cos(ϕ + 2kπ) + i sin(ϕ + 2kπ)) (4.2.107)

Draft
with k ∈ N and where ϕ is named the argument of z and is traditionally denoted by:
arg(z) (4.2.108)
The properties of the cosine and sine (see section Trigonometry) lead us directly to write for the
argument:
arg(¯z) = −arg(z) and arg(−z) = arg(z + π) (4.2.109)
We also prove among other things with the Taylor series (see section Sequences and and Series)
that:
cos(ϕ) = 1 −
ϕ2
2!
+
ϕ4
4!
− . . . + (−1)k ϕ2k
(2k)!
+ ... (4.2.110)
and:
sin(ϕ) = ϕ −
ϕ3
3!
+
ϕ5
5!
− . . . + (−1)k ϕ2k+1
(2k + 1)!
+ ... (4.2.111)
which sum is similar to:
eϕ
= 1 + ϕ +
ϕ2
2!
+
ϕ3
3!
+ . . . +
ϕk
k!
+ ... (4.2.112)
but instead perfectly identical to the Taylor expansion of eix
:
eiϕ
= 1 + iϕ −
ϕ2
2!
− i
ϕ3
3!
+ . . . + ik ϕk
k!
+ ... = cos(ϕ) + i sin(ϕ) (4.2.113)
So ﬁnally, we can write:
z = r(cos(ϕ) + i sin(ϕ)) = reiϕ
(4.2.114)
relation named Euler’s formula.
Using the properties of trigonometric functions:
cos(ϕ) + i sin(ϕ) = eiϕ
cos(ϕ) − i sin(ϕ) = e−iϕ
(4.2.115)
Depending on we sum or subtract the this gives us the Euler formulas or Moivre and Euler
formulas:
cos(ϕ) =
eiϕ
+ e−iϕ
2
sin(ϕ) =
eiϕ
− e−iϕ
2i
(4.2.116)
Note that the angle can be a purely a complex number! This is to say that in all generality
trigonometric functions can be considered as functions that go from C to C.

Draft
Thanks to the exponential form of a complex number, very commonly used in many ﬁelds of
physics and engineering, we can easily draw relations such that starting from (remember that
cis is an old notation that stands for the cos(ϕ) + i sin(ϕ) being in the parenthesis):
z = r(cos(ϕ) + i sin(ϕ)) = rcis(ϕ) = reiϕ
z1 = r1(cos(ϕ1) + i sin(ϕ1)) = r1cis(ϕ1) = r1eiϕ1
z2 = r2(cos(ϕ2) + i sin(ϕ2)) = r1cis(ϕ2) = r2eiϕ2
(4.2.117)
and assuming known the basic trigonometric identities (see section Trigonometry) we have the
following relations for the multiplication of two complex numbers:
z1z2 = r1r2 [cos(ϕ1 + ϕ2) + i sin(ϕ1 + ϕ2)] = r1eiϕ1
r2eiϕ2
= r1r2cis(ϕ1 + ϕ2) = r1r2ei(ϕ1+ϕ2)
(4.2.118)
therefore:
arg(z1z2) = arg(z1) + arg(z2) (4.2.119)
and therefore if n is a positive integer:
arg(zn
) = narg(z) (4.2.120)
For the module (norm) of the multiplication:
|z1z2| = |r1eiϕ1
r2eiϕ2
| = |r1r2ei(ϕ1+ϕ2)
| = r1r2 = |r1||r2| (4.2.121)
Therefore:
|zm
| = |z|m
(4.2.122)
For the division of two complex numbers:
z1
z2
=
r1
r2
[cos(ϕ1 − ϕ2) + i sin(ϕ1 − ϕ2)] =
r1
r2
cis(ϕ1 − ϕ2) =
r1
r2
eiϕ1
eiϕ2
=
r1
r2
ei(ϕ1−ϕ2)
(4.2.123)
The module of their division then comes immediately:
z1
z2
=
|z1|
|z2|
(4.2.124)
therefore we have for the argument:
arg
z1
z2
= arg
r1eiφ1
r2eiφ2
= arg
r1
r2
ei(ϕ1−ϕ2)
= ϕ1 − ϕ2 = arg(z1) − arg(z2) (4.2.125)
and it comes immediately:
arg(z−1
) = arg reiφ −1
= arg
1
r
eiϕ
= −ϕ = −arg(z) (4.2.126)
For the power of a complex number (or root):
zm
= rm
eimϕ
= rm
[cos(mϕ) + i sin(ϕ)] = rm
cis(mϕ) (4.2.127)

Draft
which gives us immediate a already proved previously:
|zm
| = |z|m
(4.2.128)
and for the argument:
arg(zm
) = arg reiϕ m
= arg rm
emiϕ
= mϕ = marg(z) (4.2.129)
In case we have a unit module (norm equal to 1) as z = cos(ϕ) + i sin(ϕ) we then have the
relation:
(cos(ϕ) + i sin(ϕ))m
= eiϕ m
= eimϕ
= cos(mϕ) + i sin(mϕ) (4.2.130)
named De Moivre formula.
For the natural logarithm of a complex number, we trivially have the following relation which
is discussed in the section of Analysis Analysis:
ln(z) = ln reiϕ
= ln(r) + iϕ (4.2.131)
where ln(z) is often in the complex case written Log(z) with an uppercase L.
All previous relations could of course be obtained with the trigonometric form of complex
numbers but then require some additional lines of mathematical developments.
4.2.2.7.1.1 Fresnel Vectors (phasors)
A sinusoidal variation f(t) = r sin(ωt) can be represented as the projection (see section
Trigonometry) on the vertical y-axis (imaginary axis the set C) of a rotating vector r at angular
velocity ω around the origin in the plane xOy:
Figure 4.9 – Fresnel Representation
Such a rotating vector is named Fresnel vector and can be well interpreted as the imaginary
part of a complex number given by:
r = (r) + i (r) = r(cos(ωt) + i sin(ωt)) = reiωt
= reiφ
(4.2.132)
That is to say:

Draft
Figure 4.10 – Fresnel rotating vector
We will see the phasor again explicitly in our study of wave mechanics and geometrical optics
(as part of diffraction) in the sections with the corresponding names.
4.2.2.7.2 Transformation in the plane
It is customary to represent real numbers as points on a graduated line. The algebraic
operations have their geometric interpretation on it: the addition is a translation, a multiplica-
tion a centered scaling.
In particular we can talk about the square root of a transformation. A translation of amplitude
T may be obtained as the iteration of a translation of amplitude T/2. Similarly, a scaling
of amplitu S can be achieved as iterated scaling of faction
√
S. In particular an homothety
(scaling) of a factor 9 can be composed of two homotheties (scaling) of respectively 3 (or −3).
Then we can say that the square root takes on a geometric sense. But what about the square root
of negative numbers? In particular of the square root of −1???
A scaling of factor −1 can be seen as a symmetry with respect to the origin. But if we see this
transformation in a continuous manner. Therefore a −1 scaling factor also be seen as rotation
of π rotation around the origin.
So, the problem of negative square root is simplified. Indeed, it is not difficult to break down
a rotation of π radians inot two transformations: we can repeat either a rotation of π/2 or of
−π/2. The image of 1 is the square root of −1 and i is situated on a perpendicular to the origin
at a distance 1 either up or down.
Having successfully positioned the number i it not difficult anymore to put other complex num-
bers in the Gauss plane. We can therefore associate to 2i the product of the scaling of a factor 2
(see section Euclidean Geometry) by the rotation of center O with angle of π/2, that is to say a
similitude centered at the origin. This is what we will endeavor to prove now.
Given:
z1 = x1 + iy1 = aeiα
, z2 = x2 + iy2 = beiβ
(4.2.133)
We have the following geometric transformations properties for complex numbers (see the sec-
tion Trigonometry for the properties of sine and cosine) that we can happily combine at our

Draft
discretion:
P1. The multiplication of z1 by a real number λ in the Gauss plane corresponds trivially to
a homothety of center O (the intersection of real and imaginary axis for recall...) and of
ration λ.
Indeed:
λz1 = (λa)eiα
(4.2.134)
P2. Multiplying of z1 by a complex number of unit module corresponds a rotation of center
O and of angle corresponding to the argument of z1. Indeed:
z0z1 = eiω
aeiα
= aei(α+ω)
(4.2.135)
Remark
Then we see immediately, for example, that multiplying a complex number by i
(that is to say a complex number with sin(ω) = 1, cos(ω) = 0) corresponds a
rotation of π/2
Theorem 4.1. It is interesting to notice that in vector form the rotation of center O of z1
by z0 can be written using the following matrix:
x0 −y0
y0 x0
(4.2.136)
Proof 4.1.1. We have just seen before that z0z1 is a rotation of center O of and angle ω.
We just need to write it ﬁrst in the old style:
z0z1 = (x0 + iy0)(x1 + iy1) = (x0x1 − y0y1) + i(x0y1 + y0x1) (4.2.137)
giving in vector form:
z0z1 =
x0x1 − y0y1
0
+ i
0
x0y1 + y0x1
(4.2.138)
thus the linear equivalent application is:
x0 −y0
y0 x0
x1
y1
=
x0x1 − y0x1
x0y1 + y0y1
(4.2.139)
or as well (we fall back on the rotation matrix in the plane we that we will see in the
section of Euclidean Geometry which is a remarkable result!) using:
z = r(cos(ϕ) + i sin(ϕ)) = r(cos(ϕ + k2π) + i sin(ϕ + k2π) (4.2.140)
and in the particular and arbitrary case where r is unitary (in order to have a pure rota-
tion!):
z = (cos(ϕ + k2π) + i sin(ϕ + k2π) = x0 + iy0 (4.2.141)

Draft
we have immediately (we took again the same notations for the angle as the one we we
have in the chapter Geometry):
x0 −y0
y0 x0
x1
y1
=
cos ω − sin ω
sin ω cos ω
x1
y1
=
x0x1 − y0y1
x0y1 + y0x1
=
cos(ω)x1 − sin(ω)y1
cos(ω)y1 + sin(ω)x1
(4.2.142)
Note that the rotation matrix can also be written as:
cos(ω) − sin(ω)
sin(ω) cos(ω)
= cos(ω)
1 0
0 1
+ sin(ω)
0 −1
1 0
= cos(ω)I + sin(ω)J
(4.2.143)
as well:
x0 −y0
y0 x0
= x0
1 0
0 1
+ y0
0 −1
1 0
= x0 · I + y0 · J (4.2.144)
Q.E.D.
Thus we see that the rotation matrices are not only applications but also are complex
numbers (well it was obvious from the start but we had to show it in an aesthetic and
simple way).
So, we have for usage to put that:
1 =
1 0
0 1
and i =
0 −1
1 0 (4.2.145)
or with another common notation in linear algebra:
1 =
1 0
0 1
and i =
0 −1
1 0
(4.2.146)
The field of complex numbers is isomorphic to the field of real square matrices of dimen-
sion 2 of the type:
x0 −y0
y0 x0
(4.2.147)
It is a result that we use many times in various section of this book for specific studies in
algebra, geometry and relativistic quantum physics.
P3. The multiplication of two complex corresponds to a homothety added to a rotation. In
other words, a direct similarity.
Proof 4.1.2.
z1z2 = aeiα
beiβ
= (ab)ei(α+β)
(4.2.148)
so this is indeed a similarity of ratio b and angle β.
At the opposite, the following operation:
z1z2 = aeiα
be−iβ
= (ab)ei(α−β)
(4.2.149)

Draft
will be named a retrograde linear similarity.
Otherwise, it returns trivially an already known following relation:
arg(z1z2) = arg(z1) + arg(z2) (4.2.150)
Remarks
R1. As the sum of two complex numbers z1 + z2 can not have a special simpliﬁed
mathematical notation in any form whatsoever, then we say that the resulting
quantity is equivalent to an amplitude translation.
R2. The combination of a direct linear similarity (multiplication of two complex
numbers) and an amplitude translation (sumby a third complex number) is what
we name a direct linear similarity.
Q.E.D.
P4. The conjugate of a complex number is geometrically symmetrical with respect to the axis
such that:
¯z1 = x1 − iy1 = r(cos(α) − i sin(α)) = r(cos(−α) − i sin(−α)) = re−iα
(4.2.151)
without forgetting that (basis of trigonometry):
cos(ϕ) = cos(ϕ + k2π) sin(ϕ) = sin(ϕ + k2π) (4.2.152)
This gives us a known result:
arg(¯z1) = −arg(z1) (4.2.153)
From which we get the following property:
r(cos(ϕ + π) + i sin(ϕ + π)) = r(− cos ϕ − i sin ϕ) = −r(cos ϕ + i sin ϕ) = −z1
(4.2.154)
Hence:
argz1 + π = −arg(−z1) (4.2.155)
P5. The negation of the conjugate of a complex number is geometrically its symmetrical with
respect to the imaginary axis such that:
−¯z1 = −x1 + iy1 = r(− cos α + i sin α) = r(cos(π ± α) + i sin α) (4.2.156)
Remarks
R1. The combination of the properties P4, P5 is named a retrograde similarity.
R2. The geometric operation that consist to take the inverse of the conjugate of a
complex number (that is to say ¯z−1
) is named a pole inversion.

Draft
P6. The rotation of coordinate cente c and angle ϕ is given and denoted by:
R(z1) = c + eiϕ
(z1 − c) (4.2.157)
Some explanations could be useful for some readers:
The complex c gives a point in the Gaussian plane, which will be the center of rotation.
The difference z1 −c gives the chosen radius r. The multiplication by eiϕ
is the anticlock-
wise rotation of the radius from the origin of the Gaussian plane. Finally, the addition by
c is the necessary translation to take back the rotated radius r at its original place before
the rotation (center c). Which gives schematically:
Figure 4.11 – Representation of the complex rotation
P7. On the same idea, we get and denote an homothety of center c and ratio λ by:
H(z1) = c + λ(z1 − c) (4.2.158)
Some explanations could be useful for some readers:
The difference z1 − c always gives the radius r and c a central point in the Gauss plane.
The expression λ(z1−c) gives the homothety of the radius from the origin of the Gaussian
plane, and ﬁnally by adding c gives the necessary translation for the homothety to be see
as being made from center c.

Draft
4.2.2.8 Quaternion Numbers
Also named hypercomplex quaternions numbers were invented in 1843 by William Rowan
Hamilton to generalize complex numbers.
Definition: A quaternion is an element (a, b, c, d) ∈ R4
and for which we denote by H the set
that contains it and what we name the set of quaternions.
A quaternion can also be represented in a row or column such as:
(a, b, c, d) =




a
b
c
d



 (4.2.159)
We define the sum of two quaternions (a, b, c, d) and (a , b , c , d ) by:
(a, b, c, d) + (a , b , c , d ) = (a + a , b + b , c + c , d + d ) (4.2.160)
Remark
It is the natural addition in R4
seen as a R-vector space (see section Set Theory).
The associativity is verified by applying the corresponding properties of the operations on R.
We also define the multiplication:
(a, b, c, d) · (a , b , c , d ) (4.2.161)
of two quaternions (a, b, c, d) and (a , b , c , d ) by the expression:
(a, b, c, d) · (a , b , c , d ) =




aa − bb − cc − dd
(ab + ba ) + (cd − dc )
(ac + ca ) − (bd − db )
(da + ad ) + (bc − cb )



 (4.2.162)
It may be hard to accept but we will be a little further below that there is a family resemblance
with the complex numbers.
We can notice that the law of multiplication is not commutative. Indeed, taking the definition
of the multiplication above, we have:
(0, 1, 0, 0) · (0, 0, 1, 0) = (0, 0, 0, 1)
(0, 0, 1, 0) · (0, 1, 0, 0) = (0, 0, 0, −1)
(4.2.163)
But we can also notice that:
(0, 1, 0, 0) · (0, 0, 1, 0) = −1(0, 0, 1, 0) · (0, 1, 0, 0) (4.2.164)
Remark
It is the natural addition in R4
seen as a R-vector space (see section Set Theory).

Draft
The law of multiplication is distributive with the addition law but it is an excellent example
where we must still be careful to prove the left and right distributivity, since the product is not
commutative! The multiplication is neutral element:
(1, 0, 0, 0) (4.2.165)
Indeed:
(1, 0, 0, 0) · (a, b, c, d) = (a, b, c, d) · (1, 0, 0, 0) = (a, b, d, c) (4.2.166)
Any element:
(a, b, c, d) ∈ H∗
= H − {(0, 0, 0, 0)} (4.2.167)
is inversible.
Indeed, if (a, b, c, d) is a non-null quaternion, we then have necessarily:
a2
+ b2
+ c2
+ d2
= 0 (4.2.168)
otherwise the four numbers a, b, c, d are of square null, so all zero. Given then the quaternion
(a1, b1, c1, d1) defined by:
a1 =
a
a2 + b2 + c2 + d2
b1 =
−b
a2 + b2 + c2 + d2
c1 =
−c
a2 + b2 + c2 + d2
d1 =
−d
a2 + b2 + c2 + d2
(4.2.169)
then by applying mechanically the definition of the multiplication of quaternions, we check
that:
(a, b, c, d) · (a1, b1, c1, d1) = (a1, b1, c1, d1) · (a, b, c, d) = (1, 0, 0, 0) (4.2.170)
this latter quaternion is therefore the inverse for the multiplication!
Let us prove (for general knowledge) that the field of complex numbers (C, +, ×) is a subfield
of (H, +, ×).
Remark
We could also have put this proof in the section of Set Theory because we will make use
of a lot of concepts that are have seen there but it seemed to us a little more relevant to
put instead the proof here. We expect the reader to tolerate this choice.
Given H set set of quaternions of the form (a, b, 0, 0). If H is not empty, and if (a, b, 0, 0),
(a , b , 0.0) are elements H the (H , +×) is a field. Indeed:

Draft
P1. For subtraction (and therefore the addition):
(a, b, 0, 0) − (a , b , 0, 0) = (a − a , b − b , 0, 0) ∈ H (4.2.171)
P2. The multiplication:
(a, b, 0, 0) · (a , b , 0, 0) = (aa − bb , ab + ba , 0, 0) ∈ H (4.2.172)
P3. The neutral element:
(1, 0, 0, 0) ∈ H (4.2.173)
P4. And finally the inverse:
a
a2 + b2
, −
b
a2 + b2
, 0, 0 ∈ H (4.2.174)
of (a, b, 0, 0) is still in.
Therefore (H , +, ×) is a subfield of H. Given then the application:
f : a + ib → (a, b, 0, 0)
C → H
(4.2.175)
f is bijective, and we easily check that for any complex z1, z2, we have:
f(z1 + z2) = f(z1) + f(z2)
f(z1z2) = f(z1)f(z2)
(4.2.176)
Therefore f is an isomorphism of (C, +, ×) on (H , +, ×).
This isomorphism has for interest (caused) to identify C to H and to write C ⊂ H, the laws of
addition and subtraction on H extending the already known operations of C.
Thus, by convention, we will write any element of (a, b, 0, 0) of H in the complex form a +
ib. Particularly 0 is the element (0, 0, 0, 0), 1 is the element (1, 0, 0, 0) and i and the element
(0, 1, 0, 0).
We denote by analogy and by extension j the element (0, 0, 1, 0) and k the element (0, 0, 0, 1).
The family {1, i, j, k} form a basis of all quaternions seen as a vector space on R, and we will
write:
a + bi + cj + dk (4.2.177)
the quaternion (a, b, c, d).
The notation of quaternions as defined above is perfectly suited to the multiplication operation.
For the product of two quaternions we get by developing the expression:
(a + bi + cj + dk) · (a + b i + c j + d k) (4.2.178)

Draft
16 terms that we have to identify to the original deﬁnition of multiplication of quaternions to
get the following relations:
i · j = k = −j · i
j · k = i = −k · j
k · i = j = −i · k
i2
= j2
= k2
= −1
(4.2.179)
Which can be summarized in a table:
· 1 i j k
1 1 i j k
i i −1 k −j
j j −k −1 i
j j j −i −1
(4.2.180)
We can see that the expression of the multiplication of two quaternions looks partly much like
a vector product (denoted × in this book) and dot product (denoted ◦ in this book):




(ab + ba ) + (cd − dc )
(ac + ca ) − (bd − db )
(da + ad ) + (bc − cb )



 (4.2.181)
If this is not evident (which would be quite understandable), let make a concrete example:
Example:
Given two quaternions without real part:
p = xi + yj + zk q = x i + y j + z k (4.2.182)
and u, v the vectors of R3
of respective components (x, y, z) and (x , y , z ). Then the
product:
pq = (0, u)(0, v) (4.2.183)
is equal to:
p · q = (−xx − yy − zz , yz − zy , −xz + zx , xy − yx ) = (−u ◦ v, u × v)
(4.2.184)
We can also for curiosity interest us to the general case ... Given for this two quaternions:
p = (a, u) q = (b, v) (4.2.185)
Then we have:
p · q = (a + (0, u)) · (b + (0, v))
= ab + (0, av) + (0, bu) + (−u ◦ v, u × v)
= (ab − u ◦ v, av + bu + u × v)
(4.2.186)

Draft
Definition: The center of the non-commutative field (H, +, ×) is the set of elements of H
commuting for to the law of multiplication with all the elements of H.
Theorem 4.2. The center of (H, +, ×) is the set of real numbers!
Proof 4.2.1. Give H1 is the center of (H, +, ×), and (x, y, z, t) a quaternion. We must have the
following conditions are met:
Given (x, y, z, t) ∈ H1 then for any (a, b, c, d) ∈ H we seek:
(x, y, z, t) · (a, b, c, d) = (a, b, c, d) · (x, y, z, t) (4.2.187)
which give by developing:
xa − yb − zc − td = ax − by − cz − dt
xb + ya + zd − tc = ay + bx + ct − dz
xc + za − yd + tb = az + cx − bt + dy
ta + xd + yc − zb = dx + at + bz − cy
(4.2.188)
after simplification (the first line of the previous system is equal to zero on both sides of equal-
ity):
ct − dz = 0
bt − dy = 0
bz − cy = 0
(4.2.189)
the resolution of this system gives us: So that the quaternion (x, y, z, t) is the center of H it
must be real (not imaginary parts)!
Q.E.D.
Just as for complex numbers, we can define a conjugate of quaternions:
Definition: The conjugate of a quaternion Z = (a, b, c, d) is the quaternion ¯Z =
(a, −b, −c, −d).
Just as for the complex number, we notice that:
1. First clearly that if Z = ¯Z then it means that Z ∈ R
2. That Z + ¯Z ∈ R
3. That by developping the product Z ¯Z we have:
Z ¯Z = (a, b, c, d) · (a, −b, −c, −d)
= (a2
+ b2
+c2
+ d2
, −ab + ba − cd + dc, −ac +
= (a2
+ b2
+ c2
+ d2
, 0, 0, 0) = a2
+ b2
+ c2
+ d2
∈ R
(4.2.190)

Draft
that we wil adopt, by analogy with complex numbers, as a definition of the norm (or
module) of quaternions such as:
|Z| = Z · ¯Z (4.2.191)
Therefore we also have immediately (relation which will be useful later):
|ZZ | = (ZZ )(ZZ (4.2.192)
As for complex numbers (see below), it is easy to show that the conjugation is an automorphism
of the group (H, +).
Z + Z = (a + a , −b − b , −c − c , −d − d )
= (a, −b, −c, −d) + (a , −b , −c , −d )
= ¯Z + ¯Z
(4.2.193)
It is also easy to prove that it is involutive. Indeed:
¯¯Z = (a, −(−b), −(−c), −(−d)) = (a, b, c, d) = Z (4.2.194)
But the conjugation is not a multiplicative automorphism of the field (H, +, ×). Indeed, if we
consider the multiplication of Z, ¯Z and take the conjugate:
ZZ =




(ab + ba ) + (cd − dc )
(ac + ca ) − (bd − db )
(da + ad ) + (bc − cb )



 ZZ =




−(ab + ba ) − (cd − dc )
−(ac + ca ) + (bd − db )
−(da + ad ) − (bc − cb )



 (4.2.195)
we see immediately (at least for the second row) that we have:
ZZ = ZZ (4.2.196)
Let us now back to our norm (or module) .... For this, let us calculate the square of the norm
|ZZ |:
|ZZ |2
= (ZZ ) · (ZZ ) (4.2.197)
We know (by definition) that:
ZZ =




(ab + ba ) + (cd − dc )
(ac + ca ) − (bd − db )
(da + ad ) + (bc − cb )



 (4.2.198)
Let us denote this product in such a way that:
Z · Z = (α, β, γ, λ) = Z” (4.2.199)
Then we have:
Z”Z” = α2
+ β2
+ γ2
+ λ2
(4.2.200)

Draft
substituting it comes:
(ZZ ) · (ZZ ) = (aa − bb − cc − dd )2
+ (ab + ba + cd − dc )2
+ (ac + ca − bd + db )2
+ (da + ad + bc − cb )2
(4.2.201)
after an elementary algebraic development (frankly boring) we ﬁnd:
(ZZ ) · (ZZ ) = (a 2
+ b 2
+ c 2
+ d 2
)(a2
+ b2
+ c2
+ d2
) = |Z |2
|Z|2
(4.2.202)
Therefore:
(ZZ ) · (ZZ ) = |Z|2
|Z |2
= |ZZ |2
(4.2.203)
Remark
The norm is therefore a homomorphism of (H, ×) in (R, ×). Subsequently, we will
denote by G all the quaternions of unit norm.
4.2.2.8.1 Matrix Interpretation of Quaternions
Given q and p two quaternions and given the application:
p → qp (4.2.204)
The (left) multiplication can be made with a linear application (see section Linear Algebra) on
H.
If q is written:
a + bi + cj + dk (4.2.205)
this application has for matrix in the basis 1, i, j, k:




a −b −c −d
b a −d c
c d a −b
d −c b a



 (4.2.206)
What we check well:
ZZ =




a −b −c −d
b a −d c
c d a −b
d −c b a








a
b
c
d



 =




(ab + ba ) + (cd − dc )
(ac + ca ) − (bd − db )
(da + ad ) + (bc − cb )



 (4.2.207)
In fact, we can then deﬁne the quaternions as the set of matrices with the visible structure above
if we wanted to. This will then reduce them to a sub vector space of M4(R).
Especially, the matrix of 1 (the real part of the quaternion q) is then nothing other than the
identity matrix:
M1 =




1 0 0 0
0 1 0 0
0 0 1 0
0 0 0 1



 = 1 (4.2.208)

Draft
as well:
Mi =




0 −1 0 0
1 0 0 0
0 0 0 −1
0 0 1 0



 , Mj =




0 0 −1 0
0 0 0 1
1 0 0 0
0 −1 0 0



 , Mk =




0 0 0 −1
0 0 −1 0
0 1 0 0
1 0 0 0



 (4.2.209)
4.2.2.8.2 Rotations with Quaternions
We will see now that conjugation by an element of the group G of the quaternions of
unit norm can be interpreted as a pure rotation in space!
Definition: The conjugation by a non-nul quaternion q of unit norm is the application Sq
defined on H by:
Sq : p → q · p · q−1
= q · p · ¯q (4.2.210)
and we affirm that this application is a rotation.
Remarks
R1. As q is of unit norm 1, we have obviously |q| = q¯q = 1 therefore q−1
= ¯q. This
quaternion can be seen as the proper value (of unit norm) to the application (matrix) p on
the vector ¯q (we are in a similar situation as the orthogonal rotation matrices seen in the
in section Linear Algebra).
R2. Sq is a linear application (so if it is rotation, the rotation can be decomposed into
several rotations). Indeed, let consider two quaternions p1, p2 and two real number λ1, λ2,
then we have:
Sq(λ1p1 + λ2p2) = q(λ1p1 + λ2p2)¯q = λ1qp1 ¯q + λ2qp2 ¯q = λ1Sq(p1) + λ2Sq(p2)
(4.2.211)
Let us now check that the application is indeed a pure rotation. As we saw in our study of Linear
Algebra and in particular of orthogonal matrices (see section Linear Algebra), a first obvious
condition is that the application conserves the norm.
Let us check this:
|Sq(p)| = |qp¯q| = |q||p||¯q| = |p| (4.2.212)
Moreover, we can check that a rotation of a purely complex quaternion (such that then we
restrict ourselves to R3
) and the same summed reverse rotation is zero (the vector sum up to its
opposite cancel):
Sq(p) + Sq(p) = qp¯q + qp¯q = qp¯q + q(p¯q) (4.2.213)
we trivially check that if we have two quaternions q, p then p · q = ¯q¯p since then:
Sq(p) + Sq(p) = qp¯q + q(p¯q) = qp¯q + (p¯q)¯q
= q · p · ¯q + ¯¯q · ¯p · ¯q = q · p · ¯q + q · ¯p · ¯q
= Sq(p + ¯p)
(4.2.214)

Draft
for this operation to be zero, we immediately see that we need to restrict ourselves to the purely
complex quaternions p. Since then:
Sq(p + ¯p) = Sq(0) = 0 (4.2.215)
We conclude then that p must be purely complex so the for the application Sq is a rotation and
that Sq(p) is a pure quaternion. In other words, this application is stable (in other words: a pure
quaternion by this application remains a pure quaternion).
The application Sq restricted to all purely complex quaternions is thus a vectorial isometry, that
is to say a symmetry or a rotation.
We have also seen during our study of the rotation matrices in the section of Linear Algebra and
Euclidean Geometry that such matrices should have a determinant equal to 1 so that we have a
rotation. Let’s see if this is the case of Sq:
For this, we explicitly calculate in function of:
q = a + bi + cj + dk (4.2.216)
the matrix (in the canoncial basis (i, j, k)) of Sq and we calculate its determinant. Thus we
obtain the coefﬁcients of the columns of this application by remembering that:
ij = k = −ji
jk = i = −kj
ki = j = −ik
i2
= j2
= k2
= −1
(4.2.217)
and then by calculating:
Sq(i) = (a + bi + cj + dk)i(a − bi − cj − dk)
= (ai + b(i2
) + c(ji) + d(ki))(a − bi − cj − dk)
= (ai − b − ck + dj)(a − bi − cj − dk)
= (a2i + ab − ack + adj) − (ba − b2
i − bcj − bdk)
− (cak − cbj + c2
i + cd) + (daj + dbk + cd − d2
i)
= (a2
+ b2
− c2
− d2
)i + (ab − ack + adj) − (ba − bcj − bdk)
− (cak − cbj + cd) + (daj + dbk + cd)
= (a2
+ b2
− c2
− d2
)i + 2(ad + bc)j + 2(bd − ac)k
(4.2.218)
Sq(j) = (a + bi + cj + dk)j(a − bi − cj − dk)
= (aj + b(ij) + c(j2
) + d(kj))(a − bi − cj − dk)
= (aj + bk − c − di)(a − bi − cj − dk)
= (a2
j + abk + ac − adi) + (bak − b2
j + bci + bd)
− (ca − cbi − c2
j − cdk) − (dai + db − dck + d2
j)
= (a2
− b2
+ c2
− d2
)j + (abk + ¨¨ac − adi) + (bak + bci + bd)
− (¨¨ca − cbi − cdk) − (dai + db − dck)
= 2(bc − ad)i + (a2
− b2
+ c2
− d2
)j + 2(ab + cd)k
(4.2.219)

Draft
Sq(k) = (a + bi + cj + dk)k(a − bi − cj − dk)
= (ak + b(ik) + c(jk) + d)(k2
))(a − bi − cj − dk)
= (ak − bj + ci − d)(a − bi − cj − dk)
= (a2
k − abj + aci + ad) − (abj + b2
k + bc − bdi)
+ (cai + cb − c2
k + cdj) − (da − dbi − dcj − d2
k)
= (a2
− b2
− c2
+ d2
)k + (−abj + aci + ad) − (abj + ddbc − bdi)
+ (cai + ddcb + cdj) − (da − dbi − dcj) = 2(ac + bd)i + 2(cd − ab)j + (a2
−
(4.2.220)
We must then calculate the determinant of the following matrix (pfff ...):


a2
+ b2
− c2
− d2
2(ad + bc) 2(bd − ac)
2(bc − ad) a2
− b2
+ c2
− d2
2(ab + cd)
2(ac + bd) 2(cd − ab) (a2
− b2
− c2
+ d2
)

 (4.2.221)
remembering that (which also simplifies the expression of the terms of the diagonal as we can
see in some books):
a2
+ b2
+ c2
+ d2
= 1 (4.2.222)
we find that the determinant is indeed equal to 1. Otherwise, we can check this with Maple
4.00b:
bwith@lin—lgAX
beXalin—lg‘m—trix“@QDQD‘—”PC˜”PE™”PEd”PDPB@—BdC˜B™AD
PB@˜BdE—B™ADPB@˜B™E—BdAD—”PE˜”PC™”PEd”PDPB@—B˜C™BdAD
PB@—B™C˜BdADPB@™BdE—BÃD—”PE˜”PE™”PCd”P“AY
bf—™tor@det@eAAY
Let us now show that this rotation is a half axis turn (the example that may seem particular is in
fact general!):
First, if:
q = xi + yj + zk (4.2.223)
we have:
Sq(q) = qq¯q = q (4.2.224)
which means that the axis of rotation (x, y, z) is fixed by the application Sq itself!
On the other hand, we have seen that if q is a purely complex quaternion of norm 1 then:
q−1
= ¯q and ¯q = −q (4.2.225)
Which gives us the relation:
q2
= q · (−¯q) = q · (−(q−1
)) = −1 (4.2.226)

Draft
This result leads us to calculate the rotation of a rotation:
Sq(Sq(p)) = q(qp¯q)¯q = q2
p¯q2
= Sq2 (p) = (−1)p(−q)2
= −pq2
= (−1)p(−1) = p (4.2.227)
Conclusion: Since the rotation of a rotation is a full turn, then Sq is necessarily a half-turn:
Sq(p) = −p (4.2.228)
relatively (!) to the axis (x, y, z).
At this stage, we can say that any rotation of the space can be represented by Sq (the conjugation
by a quaternion q of norm 1). Indeed, the half turns generates the group of rotations, that is to say
that any rotation can be expressed as the product of a finite number of half-turns, and therefore
by conjugation of a product of quaternions unitary norm (product which is itself a quaternion
of unitary norm...).
We will still give an explicit form connecting a rotation and the quaternion that represents it,
just as we did for complex numbers.
Theorem 4.3. Given u(x, y, z) a unit vector and θ ∈ [0, 2π] angle. The we affirm that the
rotation of axis u and angle θ corresponds to the application Sq, where q is the quaternion:
q = cos
θ
2
+ x sin
θ
2
+ y sin
θ
2
j + z sin
θ
2
k = cos
θ
2
, sin
θ
2
u




cos θ
2
x sin θ
2
y sin θ
2
z sin θ
2




(4.2.229)
For this assertion is verified, we know we need that:
• The norm of q is equal to 1
• The determinant of the application Sq is equal to 1
• The application Sq conserves the norm
• The application Sq returns all collinear vector to the axis of rotation on the axis of rotation
itself.
Proof 4.3.1. Ok let us check every point:
1. The norm of the quaternion previously proposed is indeed equal to 1:
|q| = cos2 θ
2
+ x2
sin2 θ
2
+ y2
sin2 θ
2
+ z2
sin2 θ
2
= cos2 θ
2
+ sin2 θ
2
(x2
+ y2
+ z2
)
= cos2 θ
2
+ sin2 θ
2
= 1
(4.2.230)

Draft
and as u(x, y, z) is of unit norm, we have:
x2
+ y2
+ z2
= 1 (4.2.231)
Therefore:
|q| = cos2 θ
2
+ sin2 θ
2
(x2
+ y2
+ z2
) = cos2 θ
2
+ sin2 θ
2
= 1 (4.2.232)
2. The fact that q is a quaternion of unit norm immediately leads to the fact that the deter-
minant of the application Sq is also equal to 1. We have already proved it above in the
general case of any quaternion of norm 1 (necessary and sufficient condition).
3. It is the same for the conservation of the norm. We have already proved earlier above
that this was the case anyway when the quaternion q of norm 1 (necessary and sufficient
condition).
4. Let us now prove that all collinear vector to the axis of rotation is projected onto the axis
of rotation itself. Let us denote by q the purely imaginary unitary quaternion xi+yj+zk.
Then we have:
q = cos
θ
2
+ sin
θ
2
q (4.2.233)
Then:
Sq(q ) = qq ¯q (4.2.234)
but as q is the restriction of q to the pure elements that constitute it, this is equivalent as
to write:
Sq(q ) = Sq(q) = qq¯q = q (4.2.235)
Let us now show why we choose the writing θ/2. If v = (x1, y1, z1) denotes a unit vector
orthogonal to u (therefore perpendicular to the axis of rotation), and p the quaternion
xi + yj + zk then we have:
Sq(p) = cos
θ
2
+ sin
θ
2
q p cos
θ
2
− sin
θ
2
q
cos2 θ
2
p + cos
θ
2
sin
θ
2
(q p − pq ) − sin2 θ
2
q pq
(4.2.236)
We have shown during the definition of multiplication of two quaternions that:
pq = −q p (4.2.237)
therefore we get:
Sq(p) = cos2 θ
2
p + 2 cos
θ
2
sin
θ
2
q p − sin2 θ
2
q pq
= cos2 θ
2
p + 2 cos
θ
2
sin
θ
2
q p + sin2 θ
2
q p(−q )
= cos2 θ
2
p + 2 cos
θ
2
sin
θ
2
q p + sin2 θ
2
q p¯q
(4.2.238)

Draft
We have also prove earlier above that:
Sq(p) = −p = qp¯q (4.2.239)
Therefore:
q p¯q = Sq (p) = −p (4.2.240)
(the half turn of axis (x, y, z)). So:
Sq (p ) = cos2 θ
2
p + 2 cos
θ
2
sin
θ
2
q p − sin2 θ
2
p
= cos2 θ
2
− sin2 θ
2
p + 2 cos
θ
2
sin
θ
2
q p
= cos(θ)p + sin(θ)q p
(4.2.241)
Remark
We are beginning to see here already the usefulness of having chose from the be-
ginning θ/2 for the angle!
Q.E.D.
We know that p is the pure quaternion likened to a unit vector v orthogonal to the axis of
rotation u itself equated withthe purely imaginary part of q . We notice then immediately that
the imaginary part of the product (defined!) of the quaternion q p is equal to the cross product
u × v = w. This vector product therefore generates a vector perpendicular to u, v.
The pair (v, w) thus form a plane perpendicular to the axis of rotation u (that’s as for the simple
complex numbers C in which we have the Gaussian plane and perpendicular to it the axis of
rotation!).
Then finally:
Sq(p) = cos(θ)p + sin(θ)q p = cos(θ)v + sin(θ)w (4.2.242)
We fall back with on rotation based on a plane (but therefore be in space!) identical to that
shown earlier above with the standard complex numbers C in the Gaussian plane. For more
details the reader can refer the section of Spinor Calculus.
So we know how to do any kind of rotation in space in a single mathematical operation and with
a bonus: with the free choice of the axis!
We can now better understand why the algebra of quaternions is not commutative. Indeed, the
vector rotations of the plan are commutative but those of space are not like show us the example
below:
Given the initial configuration:

Draft
Figure 4.12 – Starting situation for quaternion rotations
Then a rotation about the X-axis followed by a rotation around the Y axis:
Figure 4.13 – Example quaternion X − Y rotation
is not equal to a rotation around the Y -axis followed by a rotation about the axis X:
Figure 4.14 – Example of non-equivalence for quaternion rotation
The results will be fundamental for our understanding of spinors (see section Spinor Calculus)!
4.2.2.9 Algebraic and Transcendental Numbers
Definitions:
D1. We name algebraic integer of degree n, any complex number that is a solution of an uni-
variate algebraic equation of degree n, ie a polynomial of degree n (concept that we will
discuss in the chapter of Algebra) whose coefficients are integers and whose dominant
coefficient is equal to 1.
D2. We name algebraic number of degree n, any complex number that is a solution of an
univariate algebraic equation of degree n, ie a polynomial of degree n whose coefficients
are rational.
The set of algebraic number is sometimes denoted by Q or A.

Draft
Theorem 4.4. A first interesting result and particularly in this area of study (mathematical
curiosity ...) is that a rational number is an algebraic integer of degree n if and only if it’s
an integer (read several times need...). In scientific terms, we the say that the ring Z is fully
closed.
Proof 4.4.1. We will assume that the number p/q , where p and q are two prime integers (that
is to say that their ratio does not give an integer or more rigorously ... that the greatest common
divisor of p, q is equal to 1! , is a root of the following polynomial (see section Calculus) with
relativer integer coefficients (∈ Z) and whose dominant coefficient is equal to 1:
xn
+ an−1xn−1
+ . . . + a1x + a0 (4.2.243)
where the equality with zero of the polynomial is implicit.
In this case:
pn
= −(an−1pn−1
+ . . . + a1pqn−2
+ a0qn−1
)q (4.2.244)
Since the coefficients are by definition all integers and their multiple in the parenthesis also,
then the parenthesis has necessarily a value in Z.
Therefore, q (at the right of the parenthesis) divides a power of p (at the left of the equality),
which is possible, in the set Z (because our bracket has a value in this same set for recall...),
only if q is equal to ±1 (as they were prime together).
So among all rational numbers the only that are solutions of polynomial equations with relative
integer coefficients (∈ Z) for which the dominant coefficient is equal to 1 are relative integers!
Q.E.D.
To take another interesting and particular case, it is easy to show that any rational number is an
algebraic number. Indeed, if we take the simplest following univariate polynomial:
qx − p = 0 (4.2.245)
where p and q are relatively prime and where q is different from 1. So as this is a simple
polynomial with rational coefficients (∈ Q), after remaniment we have:
x =
p
q
(4.2.246)
So since p and q are relatively prime and q is different from 1, we have indeed that every rational
number is an algebraic number of degree 1.
We also have the real (and irrational) number
√
2 which is an algebraic integer of degree 2
because it is the root of:
x2
− 2 = 0 (4.2.247)
and the complex number i is also an algebraic integer of degree 2 because it is the root of the
equation:
x2
+ 1 = 0 (4.2.248)

Draft
etc.
Definition: A transcendental number is a real or complex number that is not algebraic. That
is, it is not a root of a non-zero polynomial equation with rational coefficients.
Theorem 4.5. The set of all transcendental numbers is uncountable. The proof is simple and
requires no difficult mathematical development.
Proof 4.5.1. Indeed, since the polynomial with integer coefficients are countable, and since each
of these polynomials has a finite number of roots (see the Factorization Theorem in the section
Calculus), the set of algebraic numbers is countable! But the argument of Cantor’s diagonal
(see section Set Theory) states that real numbers (and therefore also the complex numbers) are
uncountable, so the set of all transcendental numbers must be uncountable.
In other words, there is much more transcendental numbers than algebraic numbers...
Q.E.D.
The best known transcendent numbers are π and e. We are still looking to provide you a proof
more nice and intuitive than that of Hilbert or Lindemann–Weierstrass.
Here is a small summary of all the stuff see until now:
Figure 4.15 – Numbers Type N, Z, Q, R, C,...

Draft
4.2.2.10 Abstract Numbers (variables)
A number may be considered as doing abstraction from the nature of the objects that constitute
the group that it characterizes as well as how to codify it (Indian notation, Roman notation,
etc.). We then say that the number is an abstract number. In other words, an abstract number,
is a number that does not designate the quantity of any particular kind of thing.
Remark
Arbitrarily, the human being has adopted a numerical system mainly used in the World
and represented by the symbols 0, 1, 2, 3, 4, 5, 7, 8, 9 of the decimal system that will be
supposedly known both in writing thant orally by the reader (language learning).
For mathematicians, it is not advantageous to work with these symbols because they represent
only specific cases. What seek theoretical physicists and mathematicians are literal relations
applicable in a general case and that engineers can according to their needs change these abstract
numbers by numeric values that correspond to the problem they need resolve.
These abstract numbers today commonly named variable or unknown, used in the context
of literal calculation are very often represented since the 16th century by:
1. The Latin alphabet:
a, b, c, d, e, . . . , x, y, z; A, B, C, D, E, . . . , X, Y, Z
where the first lower case letters of the latin alphabet (a, b, c, d, e...) are often used to
represent an abstract constant, while the lowercase letters of the end of the latin alphabet
(..., x, y, z) are used to represent entities (variables or unknowns) we seek the value.
2. The Greek alphabet:
Aα Alpha Λλ Lambda
Bβ Beta Mµ Mu
Γγ Gamma Nν Nu
∆δ Delta Ξξ Xi
E ε Epsilon Oo Omicron
Zζ Zeta Ππ Pi
Hη Eta Pρ Rho
Θθϑ Theta Σσ Sigma
Iι Iota Tτ Tau
Kκ Kappa Υυ Upsilon
Φφϕ Phi Xχ chi
Ψψ Psi Ωω Omega
Table 4.10 – Greek Alphabet
which is particularly used to represent more or less complex mathematical operators (such
as the index sum Σ, the indexed product Π, the variational δ, the infinitesimal element ε,
partial differential ∂, etc.) or variables in the field of physics (as ω for the pulsation, ν for
the frequency, ρ for the density, etc.).

Draft
3. The modernized Hebrew alphabet (with less intensity...)
As we have seen, a transfinite cardinal for example is denoted by the letter aleph: N0.
Although these symbols can represent any number there are some who can represent physical
constants also named Universal constant as the speed of light c, the gravitational constant G,
the Planck constant h, the number π, etc.
We use very often still other symbols that we will introduce and define when reading this book.
Remark
The letters to represent numbers had been used for the first time by Vieta in the 16th
century.
4.2.2.10.1 Domain of a Variable
A variable is therefore likely to take different numerical values. All these values can
vary according to the character of the problem considered.
Given two numbers a and b such that a b, then: Definitions:
D1. We name domain of definition of a variable, all numerical values it is likely to take
between two specified limits (endpoints) or on a set (like N, R, R+
, etc.).
D2. We name closed interval with endpoints a and b, the set of all numbers x between these
two values and we denote as example as follows:
[a, b] = {x ∈ R | a ≤ x ≤ b} (4.2.249)
D3. We name open interval with endpoints a and b, the set of all numbers x between these
two values not included and we denote it as example as follows:
]a, b[ = {x ∈ R | a x b} (4.2.250)
D4. We name interval closed, left open right or semi-closed left the following relation as
example:
[a, b[ = {x ∈ R | a ≤ x b} (4.2.251)
D5. We name interval open left, closed right or semi-closed right the following relation
as example:
]a, b] = {x ∈ R | a x ≤ b} (4.2.252)
Or in a summary and imaged form and as often denoted in Switzerland:

Draft
Type Visual Math notation Explicitly
[a, b] a ≤ x ≤ b Closed bounded interval
[a, b[ a ≤ x b Semi-closed and bounded interval
on a and semi-open on b (or left
semi-closed and right semi-open)
]a, b] a x ≤ b Semi-open bounded interval on a
and semi-closed on b (or left semi-
open and right semi-closed)
]a, b[ a x b Bounded open interval
] − ∞, b] x ≤ b Unbounded interval closed on b (or
closed right)
] − ∞, b[ x ≤ b Unbounded interval open on b (or
open right)
[a, +∞[ a ≤ x Unbounded interval closed on a (or
closed left)
]a, +∞[ a x Unbounded interval open on a (or
open left)
Table 4.11 – Resume of main Combinatorial Analysis cases
and according to the international norm ISO 80000-2: 2009 (since Switzerlant has the art not
respecting internation norms and standards):
Type Visual Math notation Explicitly
[a, b] a ≤ x ≤ b Closed bounded interval
[a, b) a ≤ x b Semi-closed and bounded interval
on a and semi-open on b (or left
semi-closed and right semi-open)
(a, b] a x ≤ b Semi-open bounded interval on a
and semi-closed on b (or left semi-
open and right semi-closed)
(a, b) a x b Bounded open interval
(−∞, b] x ≤ b Unbounded interval closed on b (or
closed right)
(−∞, b[ x ≤ b Unbounded interval open on b (or
open right)
[a, +∞) a ≤ x Unbounded interval closed on a (or
closed left)
(a, +∞) a x Unbounded interval open on a (or
open left)

Draft
Remarks
R1. The notation {x such thata x b} denotes the set of real numbers x strictly
greater than x and strictly less than b.
R2. To fact that an interval is for example opened on b means that the real number b is
not part thereof. By cons, if it had been closed then b would be part of it.
R3. If the variable x can take all possible negative and positive values we write
therefore: ]−∞, +∞[ where the symbol ∞ means infinite. Obviously there can be
combinations of open infinite right intervals with left endpoint and vice versa.
R4. We will recall some of these concepts with a different approach when studying
Algebra (literal calculation).
We say that the variable x is an ordered variable if by representing its domain of definition
by a horizontal axis where each point on the axis represents a value of x, then for each pair of
values, we can say that that there is an antecedent and one that is a subsequent. Here the
notion of antecedent and subsequent is not related to the concept of time it expresses just how
the values of the variable are ordered.
Definitions:
D1. A variable is say to be increasing if each subsequent value is greater than each an-
tecedent value.
D2. A variable is say to be decreasing if each subsequent value is smaller than each an-
tecedent value.
D3. The increasing and decreasing variables are named variables with monotinc variations
or simply monotonic variables.
∼90 %
31 votes, 69.68%

Draft
4.3 Arithmetic Operators
Talking about numbers like we did in the previous section naturally leads us to consider the
operations of calculus. It is therefore logic that we make a non-exhaustive description of the
operations that may exist between the numbers. This will be the goal of this section.
We will consider in this book that there are two types of key tools in arithmetics (we do not
speak of algebra but arithmetic!):
• Arithmetic operators:
There are two basic operators (addition + and subtraction −) from which we can build
other operators: the multiplication (whose contemporary symbol × was introduced in
1574 by William Oughtred) and the division (whose old symbol was ÷ but since end
of the 20th century we use simple the slash symbol).
These four operators are commonly named rational operators. We will see them more
in details after setting the binary relations.
Remark
Rigorously addition could be enough if we consider the common set of real number
R because therefore the subtraction is only the addition of a negative number.
• Binary operators (relations):
There are six basic binary relations (equal =, different =, greater than , less than ,
greater or equal ≥, less than or equal ≤) that compare the order of amplitude of elements
that are on the left and on the right of these relations (thus at the number of two, hence
the name binary) in order to draw some conclusions. The majority of binary relations
symbols were introduced by Vieta and Harriot in the 16th century..
It is obviously essential to know as best a possible these tools and their properties before going
through into more strenuous calculations.
4.3.1 Binary Relations
Deﬁnitions:
D1. Consider two non-empty sets E and F (see section Set Theory) not necessarily identical.
If to some given elements x of A we can associate with a precise mathematical rule R
(unambiguous) one element y of F, we deﬁne therefore a functional relation that maps
E to F and that we write:
R : E → F (4.3.1)

Draft
Thus, more generally, a functional relation R can be defined as a mathematical rule that
associates to given components x of E, some given elements y of F.
So, in this more general context, if xRy, we say that there y is an image of x through R
and that x is a precedent or preimage of y.
The set of pairs (x, y) such that xRy is a true statement generates a graph or represen-
tation of the relation R. We can represent these couples in a proper chosen way to make
a graphical representation of the relationship R.
This is a type of relation on which we will come back in the section Functional Analysis
under the form: R : f(x) = yof and that does not interest us directly in this section.
D2. Consider a non-empty set E, if we associate with this set (and only to this one!) tools to
compare its items between them when we talk about a binary relation or comparison
relation and that we write for any element x and y of A:
xRy (4.3.2)
These relationships can also most of time be presented graphically. In the case of con-
ventional binary operators comparison where A is the set of natural numbers N, relative
Z, rationals Q or real R, that is graphically represented by a horizontal line (typically...);
in the case of congruence (see section Number Theory) it is represented by lines in the
plane whose points are given by the constraint of congruence.
4.3.1.1 Equalities
It is difficult to define the term equality in a general case applicable to any situation. For
our part, we will allow ourselves for this definition to take the inspiration of the extensionality
theorem of Set Theory (discussed later in another section).
Definitions:
D1. Two elements are equal if and only if they have the same values. The strict equality is
described by the symbol = that therefore means equal to (this symbol was introduced
in 1557 by Robert Rocorde).
If we have a = b and c is any given number (or vector/matrix) and any operation (such
as addition, subtraction, multiplication or division) then:
a c = b c (4.3.3)
This property is used to solve or simplify any type of equations.
Obviously we have (property of reflexivity):
a = b ⇔ b = a (4.3.4)
And also (property of transitivity):
a = c
b = c
a = b
We will not enumerate the other properties of the equaliy in the section (for more details
see the section Set Theory).

Draft
D2. If two elements are not strictly equal, that is to say inequal..., we are connecting them
by the symbol = and we say they are not equal.
If we have a b or a b then:
a = b (4.3.5)
There are still other equality symbols, which are an extension of two we have defined previously.
Unfortunately, they are often misused (we could say rather that they are used in the wrong
places) in most of the books available on the market (and this book is not an exception):
1. ∼=: Should be used for congruence but in fact is mostly used to indicate an approxmation.
2. ≈: Should be use for approximations but in fact ∼= use instead.
3. ≡: Is used to say that two elements are equivalent but in practice most people use =.
4. :=: Is used to say that one element is by definition equal to another one.
5.
.
=: Should be used to say equal by definition to but in fact people use instead :=.
6. ∼: Is use most of time in Statistics to say follows the law... but some practitioners use
instead = or to say asymptotically equal.
4.3.1.2 Comparators
The comparators are tools that allow us to compare and order any pair of numbers (and also
Sets!).
The possibility of order numbers is fundamental in mathematics. Otherwise (if it was not pos-
sible to order), there would be a lot of things that would shock our habits, for example (some of
the concepts presented in the following sentence have not yet been presented but we would still
make reference to them): no more monotonic functions (especially sequences) and linked to it
the derivation would therefore indicate nothing more about the variation direction, no more
approach of roots of polynomial by dichotomy (classical research algorithm in an ordered set
that split in two at each iteration), no more segments in geometry, no more than half space, no
more convexity, we can not oriented space anymore, etc. It is therefore important to be able to
order things as you can see...!
Thus, for any a, b, c ∈ R we write when a is greater than or equal to b:
a b (4.3.6)
and when a is less than or equal to b:
a b (4.3.7)
Remark
It is useful to recall that the set of real numbers R is a totally ordered group (see section
Set Theory), otherwise we could not establish order relations among its elements (which
is not the case for complex numbers C that we can not order!).

Draft
Deﬁnition: The symbol ≤ is an relation order (see the rigorous deﬁnition further below!)
which means less than or equal to and conversely the symbol ≥ is also an order relation that
means greater than or equal to.
We also have relatively to the strict comparison the following properties that are relatively intu-
itive:
a b and b c ⇒ a c
a b and b c ⇒ a c
(4.3.8)
and:
a b and b = c ⇒ a c
a b and b = c ⇒ a c
(4.3.9)
if:
a b and c 0 ⇒ ac bc (4.3.10)
if:
a b ⇒ a + c b + c and a b ⇒ a − c b − c (4.3.11)
and vice versa:
a b ⇒ a + c b + c and a b ⇒ a − c b − c (4.3.12)
We also have:
0 a b ⇒
1
a

1
b
(4.3.13)
and vice versa:
b a 0 ⇒
1
a

1
b
(4.3.14)
We can obviously multiply, divide, add or subtract a term from each side of the relationship as
it is always true. Notive, however, that if you multiply both sides by a negative number it will
obviously change as the comparator such that:
a b and c 0 ⇒ ac bc (4.3.15)
and vice versa:
a b and c 0 ⇒ ac bc (4.3.16)
We also have:
0 a b and p ∈ R∗+
⇒ ap
bp
(4.3.17)
Consider now that b a 0 and p ∈ N∗
. Then if p is an even integer:
0 ap
bp
(4.3.18)

Draft
else if p is odd:
ap
bp
(4.3.19)
This result simply comes from the multiplication of signs rule since the power when not frac-
tional is only a multiplication.
Finally:
0 a b and n ∈ N∗
⇒ n
√
a
n
√
b (4.3.20)
The relations
(4.3.21)
thus correspond respectively to: (strictly) greater than, (strictly) smaller than, smaller or equal,
greater or equal, much bigger than, much smaller than.
These relations can be defined in a little more subtle and rigorous way and apply not only to
comparators (see for example the congruence relation in the section of Set Theory)!
Let us see this (the vocabulary that follows is also defined in the section of Set Theory):
Definition: Given a binary relation R of a set A to itself, a relation R on A is a subset of
the cartesian product R ⊆ A × A (that is to say, the binary relation generates a subset by the
constraints it imposes on the elements of A satisfying the relationship) with the property of
being:
P1. A reflexive relation if ∀x ∈ A:
xRx (4.3.22)
P2. A symmetrical relation if ∀x, y ∈ A:
xRy ⇒ yRx (4.3.23)
P3. An anti-symmetrical relation if ∀x, y ∈ A:
(xRy and yRx) ⇒ x = y (4.3.24)
P4. A transitive relation if ∀x, y, z ∈ A:
(xRy and yRz) ⇒ xRz (4.3.25)
P5. An connex relation if ∀x, y ∈ A:
∀x, y ∈ A ⇒ xRy oryRx (4.3.26)
Mathematicians have given special names to the families of relations satisfying some of these
properties. Definitions:

Draft
D1. A relation is named strict order relation if and only if it is only transitive (some specify
then that it is necessarily antireflexive but this last fact is then obvious...).
D2. A relation is named a preorder if and only if it is reflexive and transitive.
D3. A relation is named an equivalence relation if and only if it is reflexive, symmetric, and
transitive.
D4. A relation is named order relation if and only if it is reflexive, transitive and antisym-
metric (thus the relations , are not order relations because obviously not reflexive
relations).
D5. A relation is named total order relation if and only if it is reflexive, transitive, connex
and antisymmetric.
For the other combinations it seems (as far as we know) that there are no special name among
the mathematicians ...
Remark
The binary relations have all similar properties in natural sets N, relative Z,rational Q
and real R (there is no natural order relation on the set of complex numbers C).
If we summarize:
BINARY RELATION = =
reflexive yes no no no yes yes
symmetric yes yes no no no no
transitive yes no yes yes yes yes
connex no no no no yes yes
antisymetric yes no no no yes yes
Table 4.13 – Binary Relations
Thus we see that the binary relations ≤, ≥ form with the previously mentioned sets, total order
relations and it is very easy to see which binary relations are partial, total or equivalence order
relations.
Definition: If R is an equivalence relation on A. For ∀x ∈ A, the equivalence class of x is by
definition the set:
[x] = {y ∈ A : xRy} (4.3.27)
[x] is therefore a subset of A (x ⊆ A) which we denote also thereafter ... R (so be careful not to
confuse in what follows the equivalence relation and the subset itself...).
We thus have a new set that is named the set of equivalence classes or quotient set denoted
in this book by A/R. So:
A/R = {[x]|x ∈ A} (4.3.28)

Draft
You should know that in A/R we do not look anymore at [x] as a subset of A, but as an element!
An relation of equivalence, presented in a popularized manner... thus serves to stick one unique
label to items that satisfy the same property, and to confuse them with the said label (knowing
what we do with this label).
Example:
In the set of integers Z, if we study the remains of the division of number by 2,
we have that the result is always 0 or 1.
The zero equivalence class is then named the set of even integers numbers, the one
equivalence class is therefore named the set of odd integers. So we have two classes
of equivalence for two partitions of Z (always keep in mind this simple example for
theoretical elements that follow it helps a lot!).
If we name the first 0 and the second 1, we fall back on the operation rules between odd
and even numbers:
0 + 0 = 0 0 + 1 = 0 1 + 1 = 0 (4.3.29)
which respectively means that the sum of two even integers is even, that the sum of an
even and an odd integer is odd and that the sum of two odd integer is even.
And for the multiplication:
0 × 0 = 0 0 × 1 = 0 1 × 1 = 1 (4.3.30)
which respectively means that the two product of two even integer is even, the product
of an even and an odd integer is even and that the product of two odd integer is odd.
Now, to verify that we are dealing with an equivalence relation, we should still check that
it is reflexive (xRx), symmetrical (if xRy then yRx) and transitive (if xRy and yRz then
xRz). We will see how to check it a few paragraphs further below because this example
is a very special case of congruence relation.
Definition: The application f : A → A/R defined by x → [x] is named canonical projection.
Any element z ∈ [x] is therefore named class representative of [x].
Theorem 4.6. Now consider a set E. Then we propose to proved that there is correspondence
between the set of equivalence relations on E and all partitions of E. In other words, this
theorem says that an equivalence relation on E is nothing more but a partition on E (this is
intuitive).
Proof 4.6.1. Let R be an equivalence relation on E. We choose I = E/R as set partition
indexing and all we ask for any [x] ∈ E/R, E[x] = [x].
We just have to check the following two properties of the definition of partitions to show that
the family E[x] I
is a partition of E:

Draft
P1. Given [x], [y] ∈ E/R such that [x] = [y] then (obvious) E[x] ∩ E[y] = ∅.
P2. E =
[x]∈E/R
is obvious because if x ∈ E then x ∈ [x] = E[x].
Q.E.D.
Again, it should by easy to check with the practical example of the division by 2 given previ-
ously that the partition of even and odd numbers satisfies these two properties (if not reader can
contact us we will add this as an example).
We have therefore associated to the equivalence relation R a partition E. Conversely, if (Ei)I
is a partition of E then we almost easily verify that the relation R is defined by xRy if and only
if there exists i ∈ I such as x, y ∈ Ei is an equivalence relation! Both applications are thus
bijective and the inverses of each other.
Example:
We will now apply an example a little less trivial than the last we have seen to
the construction of rings Z/Z after a few reminders equation (for the concept of ring see
the section Set Theory).
Reminders:
1. Given two numbers n, m ∈ Z. We say that n divides m and we write n|m if
and only if there exists an integer k ∈ Z such as m = kn (see section Numbers
Theory).
2. Given d ≥ 1 is an integer. We define the relation R by nRm if and only if d|(n−m)
or in other words nRm if and only if there exists d ∈ Z such that n = m + kd.
Usually we write this n ≡ m (modulo d) instead of nRm and we say that n is
congruent to m modulo d. Remember also that n ≡ 0 (modulo d) if and only if d
divides n (see section Numbers Theory).
We will now introduce an equivalence relation on Z. Let us prove that for any integer
d ≥ 1, the congruence modulo d is an equivalence relation on Z (we have already proved
this in the section of Number Theory in our study of congruence but let us redo this work
for the fun...).
To prove this we simply have to control the three properties of the equivalence relation:
P1. Reflexivity: n ≡ n since n = n + 0d.
P2. Symmetry: If n ≡ m then n = m + kd and therefore m = n + (−k)d that is to say
m ≡ n.
P3. Transitivity: If n ≡ m and then mj then n = m + kd and m = j + k d therefore
n = j + (k + k )d that is to say n ≡ j.

Draft
In the above situation, we denote by Z/dZ the set of equivalence classes and we will
deonte by [n]d the equivalence class of congruence of a given integer n given by:
[n]d = {. . . , n − 2d, n − d, n, n + d, n + 2d, n + 3d, . . .} (4.3.31)
(each difference of two values in the braces is divisible by d and this is therefore an
equivalence class), thus:
Z/dZ = {[0]d, [1]d, [2]d, . . . , [d − 1]d} (4.3.32)
In particular (trivial since we obtain thus the all Z):
Z/2Z = {[0]2, [1]2} (4.3.33)
Remark
The operations of addition and multiplication on Z deﬁne also the operation of addition
and multiplication on Z/dZ. Then we say that these operations are compatible with the
equivalence relation and then form a ring (see section Set Theory).

Draft
4.3.2 Fundamental Arithmetic Laws
As we have said before, there is a fundamental operator (addition) from which we can define
multiplication, subtraction (provided that the chosen Numbers Set is adapted to it....) and divi-
sion (provided that the chosen Numbers Set is also adapted to it....) and around which we can
build the entire Analytical Mathematics.
Obviously there are some subtleties to be considered when the level of rigor increase. The
reader can then refer to the section of Set Theory where fundamental laws are redefined more
accurately than what will follow.
4.3.2.1 Addition
Definition: The addition of integers is an operation denoted + which has for only purpose
to bring together in one number all the units contained in several others. The result of the
operation is named the sum, the total or cumul. The numbers to be added are named
therefore terms of the addition.
Remark
The signs of addition + and subtraction − are due to the German mathematician
Johannes Widmann (1489).
Thus, A + B + C... are the terms of the addition and the result is the sum of the terms of the
addition.
Or in schematic form of a special case:
0 + 4 + 3 = 4 + 3 = 7

Draft
Figure 4.16 – One possible schema for addition
Here is a list of some intuitive properties that we assume without proofs (as in fact they are
axioms) of the operation of addition:
P1. The sum of several numbers do not depend on the order of terms. Then we say that the
addition is a commutative operation. This means concretely for any two numbers:
A + B = B + A (4.3.34)
P2. The sum of several numbers does not change if we replace two or more of them by their
intermediate result. Then we say that the addition is an associative operation:
(A + B) + C = A + (B + C) (4.3.35)
P3. The Zero is the neutral element of addition because any number added to zero gives that
number:
A + 0 = 0 (4.3.36)
P4. Depending on the set in which we work (Z, Q, R, ...), the addition may include a term in
such a way that a sum is zero. Then we say that there exists an opposate to the sum
such as:
A + ¯A = 0 (4.3.37)
We have deﬁne more rigorously the addition using the Peano axioms in the particular case of
all natural numbers N as we have already see in the section Numbers. So, with these axioms it
is possible to prove that there exists one and only one application (uniqueness), denoted + of
N × N in N satisfying:
∀n ∈ N, n + 0 = n
∀p ∈ N, ∀q ∈ N, p + s(q) = s(p + q)
∀n ∈ N, s(n) = n + 1
(4.3.38)
where S means successor.
Remark
As this book has not be written for mathematicians, we will pass the proof (relatively
long and of little interest in the case of business) and we will assume that the application
+ exists and is unique ... and that it follows from the above properties.

Draft
Let x1, x2, ..., xn be any numbers then we can write the sum as following:
x1 + x2 + . . . + xn =
n
i=1
xi (4.3.39)
by deﬁning upper and lower bound to the indexed sum (below and above the upercase greek
symbol Sigma).
Here are some properties relatively to this condenses notation that should be obvious (if not the
reader can send us a request we will add the details):
n
i=1
kxi = k
n
i=1
xi
n
i=1
k = nk
n
i=1
(xi + yi) =
n
i=1
xi +
n
i=1
yi (4.3.40)
where k is a constant.
Let us see now some concrete examples of additions of various simple number in the purpose
to practice the basis:
Examples:
The addition of two numbers relatively small is quit easy since we have learn by
heart to count to a number resulting of the operation. Therefore (examples taken on
decimal basis):
5
+ 2
7
(4.3.41)
and:
1 0
+ 3
1 3
(4.3.42)
and:
1 0 1 4
+ 3
1 0 1 7
(4.3.43)
For more bigger number we can adopt another method that human must also learn by
heart. For example:
9 2 4 4
+ 3 4 7 5
?
(4.3.44)
The algorithm (process) is therefore the following: We add the columns (4 columns in
this example) from right to left. For the ﬁrst column we have therefore 4 + 5 = 9 this
gives:

Draft
9 2 4 4
+ 3 4 7 5
9
(4.3.45)
and we continue like this for the second column where we have 4 + 7 = 11 at the
difference that now we have a number 10, then we report the ﬁrst left digit on the next
(left) column for the addition. Therefore:
9 2+1
4 4
+ 3 4 7 5
1 9
(4.3.46)
The third column we be calculated therefore as 1 + 2 + 4 = 7 which give us:
9 2+1
4 4
+ 3 4 7 5
7 1 9
(4.3.47)
For the last column we have 9 + 3 = 12 and once again we report the ﬁrst digit from the
left on the next column of the addition. Therefore:
+1
9 2+1
4 4
+ 3 4 7 5
2 7 1 9
(4.3.48)
Finally:
+1
9 2+1
4 4
+ 3 4 7 5
1 2 7 1 9
(4.3.49)
This example show how we can proceed for the addition of any real numbers: we do an addition
column by column from the right to the left and if the result of one addition is greater than 10,
we report the left digit on the next (left) column.
This algorithm (process or methodology) of addition is quite simple to understand and to exe-
cute. We will not go further on this subject add this day.

Draft
4.3.2.2 Subtraction
Deﬁnition: Subtraction is a mathematical operation that represents the operation of removing
objects from a collection. More formally the subtraction of the number A by the number B
denoted by the symbol − consist in founding the number C such that added to B gives A.
Remark
As we saw it in the section of Set Theory the subtraction in the set N could be possible
only if A B.
Formally we write an inline literal subtraction in the form:
A − B = C (4.3.50)
That must satisﬁes:
A = B + C (4.3.51)
Or in schematic form of a special case:
10 − 3 − 4 = 7 − 4 = 3
Figure 4.17 – One possible schema for subtraction
Here are some intuitive properties that we assume without proof for the subtraction operation
(as it can be deduce from the addition...):
P1. The subtraction of several numbers depends on the order of the terms. We say when than
subtraction is a non-commutative operation. Indeed:
5 − 2 = 2 − 5 (4.3.52)
P2. The subtraction of several numbers change if we replace two or more of them by their
intermediate result. We say when the subtraction is a non-associative operation. Indeed:
5 − (3 − 2) = (5 − 3) − 2 (4.3.53)
P3. The zero is not the neutral element of subtraction. Indeed, any number to which we
subtract zero gives the same number, so zero is neutral on the right ... but not left because
any number we subtract to zero does not give zero! We then say that the zero is only
neutral on the right in the case of subtraction. Indeed:
0 − 5 = 0 (4.3.54)

Draft
In most complicated cases we have a special vocabulary:
−1
7 0 4
5 1 2
1 9 2
←− carry
←− Minuend
←− Subtrahend
←− Rest or Diﬀerence
(4.3.55)
The minuend is 704, the subtrahend is 512. The minuend digits are m3 = 7, m2 = 0 and
m1 = 4. The subtrahend digits are s3 = 5, s2 = 1 and s1 = 2. Beginning at the one’s place, 4 is
not less than 2 so the difference 2 is written down in the result’s one place. In the ten’s place, 0
is less than 1, so the 0 is increased by 10, and the difference with 1, which is 9, is written down
in the ten’s place. The American method corrects for the increase of ten by reducing the digit in
the minuend’s hundreds place by one. That is, the 7 is struck through and replaced by a 6. The
subtraction then proceeds in the hundreds place, where 6 is not less than 5, so the difference is
written down in the result’s hundred’s place. We are now done, the result is 192.
Let us see now some concrete examples of additions of various simple number in the purpose
to practice the basis:
Example:
The subtraction of two relatively small numbers is pretty easy once we memorized to
count to at least the number resulting from this operation. So:
5
− 2
3
(4.3.56)
and:
1 0
− 3
7
(4.3.57)
and:
1 0 1 4
− 3
1 0 1 1
(4.3.58)
For larger numbers another possible method must be learned by heart (as well as for the
addition). For example:
4 5 7 4
− 3 7 8 5
?
(4.3.59)

Draft
we subtract the columns (4 columns in this example) from right to left. In the ﬁrst column
we have 4 − 5 = −1 0 so we report −1 to the next column (second one) and we write
10 − 1 = 9 below the horizontal line of the ﬁrst column:
4 5 7−1
4
− 3 7 8 5
9
(4.3.60)
and we continue as well for the second column 7 − 8 = −1 0 so that we report −1
on the next column (third one) and as −1 − 1 = −2 we report 10 − 2 = 8 below the
horizontal bar of the second column:
4 5−1
7−1
4
− 3 7 8 5
8 9
(4.3.61)
The third column is calculated as 5 − 7 = −2 0 and we report −1 on the next column
(fourth one) and as −1−2 = −3 we report 10−3 = 7 below the line of the third column
bar:
4−1
5−1
7−1
4
− 3 7 8 5
7 8 9
(4.3.62)
In the last column we have 4 − 3 = 1 0 therefore we report the nothing on the next
column and as 1 − 1 = 0 we report 0 below the line of the fourth column bar:
4−1
5−1
7−1
4
− 3 7 8 5
0 7 8 9
(4.3.63)
That’s how we therefore we proceed to subtracting any numbers. We make a subtraction by
column from the right to the left and if the result is a subtraction is less than zero we report −1
to the next column and the addition of the latest report on the subtraction obtained below the
line.
We have when we mix the addition and subtraction the following resulting relation that should
be obvious for most readers:
a + (b − c) = (a + b) − c
a − (b + c) = (a − b) − c
a − (b − c) = (a − b) + c
a − b = (a − c) − (b − c)
(4.3.64)
The methodology used for subtraction being based on exactly the same rules that for addition
we will expand the subject more as this seems actually useless in our point of view. This method
is very simple and of course requires some habits to work with numbers to be fully understood
and mastered.

Draft
4.3.2.3 Multiplication
Definition: The multiplication of numbers is an operation that has for purpose, given two num-
bers, one named multiplier m, and the other multiplicand M, to find a third number named
product P that is the sum (multiplication is only a successive number of sums!) as many
equal numbers to the multiplicand as there are units multiplier:
m × M = M
(1)
+ M
(2)
+ M
(1)
+... + M
(m)
=
m
i=1
M = P (4.3.65)
The multiplicand and multiplier are named product factors.
The multiplication is indicated in kindergarten by the symbol × of of the elevated dot symbol
in higher classes · or even when there is no possible confusion... without anything:
a × b = a · b = ab (4.3.66)
We can define the multiplication using the Peano axioms in the special case of natural numbers
N as we have already mentioned in the sectionNumbers. Thus, with these axioms it is possible
to prove that there is (exists) one and only one (unique) application, denoted × or more often
· of of N2
to N satisfying:
∀n ∈ N, n · 0 = 0
∀p ∈ N, ∀q ∈ N, p(q + 1) = pq + q
(4.3.67)
Remark
As this book has not be written for mathematicians, we will pass the proof (relatively
long and of little interest in the case of business) and we will assume that the application
× exists and is unique ... and that it follows from the above properties.
The power is a specific notations of a special case of the multiplication. When to multipli-
cand(s) and the multiplier(s) are typically identicals in numerical values, we denote therefore
the multiplication by (for example):
n · n · n · n · n · n · n · n = n8
(4.3.68)
This is what we name the power notation or ¨NewTermexponentations. The number in su-
perscript is what we the name the power or the exponant of the number. The notation with
exponants is said to be see for the first time in a book of Chuquet in 1484.
You can check by yourself that is properties are the following (for example):
nx
ny
= nx+y
(4.3.69)
and also:
ax
bx
= (ab)x
(4.3.70)
Here are some obvious properties about the multiplication that we will admit without proof (this
is a Set properties point of view listing):

Draft
P1. The multiplication of several numbers does not depend on the order of terms. Then we
say that multiplication is a commutative operation.
P2. The multiplication of several numbers does not change if we replace two or more of
them by their intermediate result. We then say that the multiplication is an associative
operation.
P3. The unit is the neutral element of the multiplication as any multiplicand multiplied by the
multiplier 1 is equal to the multiplicand itself.
P4. The multiplication may have a term such that the product is equal to unity (the neutral
element). Then we say that there exists a multiplicative inverse (but this depends strictly
speaking in what set of numbers we work as in some the concept of decimal number does
not exist!).
P5. Multiplication is distributive, that is to say:
a · (b + c) = ab + ac (4.3.71)
the reverse being named factorization.
Let us also introduce some special notations for the multiplication:
1. Given any numbers x1, x2, ..., xn (not necessarily equal) then we can write the product as
following:
x1 · x2 · . . . · xn =
n
i=1
xi (4.3.72)
by defining upper and lower bounds to the indexed product (above and below the upper-
case Greek letter Pi).
We trivially have respectively to the latter notation (on request we can detail more...):
n
i=1
kxi = kn
n
i=1
xi (4.3.73)
for any number k such that:
n
i=1
k = kn
(4.3.74)
We also have for example:
n
i=1
(x + y) = (x + y)n
(4.3.75)
2. We define the factorial simply (simply... because it exists also a more complex way
of defining it through the Euler Gamma function as it is done in the section of Integral
and Differential Calculus) by:
1 × 2 × 3 × 4 × · · · × n = n! (4.3.76)

Draft
with the special fact that (only the complex deﬁnition mentioned before can make this
fact obvious...):
0! = 1 (4.3.77)
Let us see some simple examples of basic multiplications:
Example:
E1. The multiplication of two relatively small numbers is fairly easy once we
have memorized count to at least the number resulting from this operation. So:
5
× 2
1 0
10
× 3
30
1 0 1 4
× 3
3 0 4 2
(4.3.78)
E2. For much larger numbers we must adopt another method that has to be memorized.
For example:
4 5 7 4
× 8
3 2
5 6
4 0
3 2
(4.3.79)
This methodology is very logical if you understand how we build a a number in base ten.
Thus we have (we’ll assume that the distributive property is mastered):
8 × 4574 = 8 × (4 · 103
+ 5 · 102
+ 7 · 101
+ 4 · 100
)
= 8 × 4000 + 8 × 500 + 8 × 70 + 8 × 4
= 32 · 103
+ 40 · 102
+ 56 · 101
+ 32 · 100
= 36592
(4.3.80)
To avoid overloading the notations in the multiplication by the vertical method, we do
not represent the zeros that would overload unnecessarily the calculations (and even more
if the multiplier and / or the multiplicand are very large numbers).

Draft
4.3.2.4 Division
Definition: The division of integers (to start with the simplest case ...) is an operation, which
aims, given two integers, one named dividend D, the other named divider d , to find a third
number named quotient Q which is the largest number whose product by the divisor can be
subtracted (so the division result of the subtraction!) the dividend (the difference being named
the rest R or sometimes the congruence).
Remark
In the case of real numbers there are never any rest at the end of the division operation
(because the quotient multiplied by the divisor gives always exactly the dividend)!
Generally in the context of integers (or algebraic equation division), if we denote by D the
dividend and by d the divisor, the quotient Q and the remainder R we have the relation:
D = Q · d + R (4.3.81)
knowing that the division was initially written as follows:
D : d =
D
d
(4.3.82)
We indicate the operation of division by placing between the two numbers, the dividend and the
divider, a symbol : or a slash / or even in kindergarten with the symbol ÷.
We refer also often by the term fraction (instead of quotient), the ratio of two numbers or in
other words, the division of the first by the second.
Remark
The sign of division : is said to be due to Gottfried Wilhelm Leibniz. The slash symbol
could have been see for the first time in the works of Leonardo Fibonacci (1202) and is
probably due to the Hindus.
If we divide two numbers and we want an integer as quotient and as remainder (if there is one
...), then we speak of euclidiean division.
For example, dividing a cake, is not a Euclidean division because the quotient is not an integer,
except if one takes the four quarters ...:

Draft
Figure 4.18 – Schematic example of a division (fractions)
If we have:
D : d = D ·
1
d
= D · iD (4.3.83)
we name iD the inverse of the dividend. At any number is associated an inverse that satisfies
this condition. From this definition it comes the notation (with x being any number other than
zero)
x
x
= x1
· x−1
= x1−1
= x0
= 1 (4.3.84)
In the case of two fractional numbers, we say they are inverse or reciprocal, when their
product is equal to unity (as the previous relations).
Remarks
R1. A division by zero is what we name a singularity. That is to say the result of the
division is: undetermined!!
R2. When we multiply the dividend and the divisor of a division (fraction) by a
same number, the quotient does not change (this is an: equivalent fraction), but the
remainder is multiplied by that number.
R3. Divide a number by a product made of several factors is equivalent to divide this
number successively by each of the factors of the product and vice versa.
R4. Fractions that are greater than 0 but less than 1 are named proper fractions. In
proper fractions, the numerator is less than the denominator. When a fraction has a
numerator that is greater than or equal to the denominator, the fraction is an improper
fraction. An improper fraction is always 1 or greater than 1. And, finally, a mixed
number is a combination of a whole number and a proper fraction.
The properties of the divisions with the condensed power notations (exponentiation) are typi-
cally as example (we will leave to the reader the fact to check this up to with numerical values):
x · x · x
y · y
= x3
· y−2
(4.3.85)

Draft
or obviously another example:
x · x · x
x · x
=
x
x
·
x
x
· x = 1 · 1 · x = x3
· x−2
= x1
= x (4.3.86)
We therefore deduce that:
xp
xq
= xp−q
(4.3.87)
Let us recall that a prime number (relative integer Z) is a number greater than 1 that has for
divisors only itself and unity (remember that 2 is prime for example). Therefore any number
that is not prime has at least one prime number as a divisor (except 1 by deﬁnition!). The
smallest divisors of an integer is a prime number (we will detail the properties of prime numbers
relatively to the operation of division in the section Numbers Theory).
Let us see some properties of the division (some of us are already known because they arise
from logical reasoning of the multiplication properties):
a
b
=
c
d
⇔ a · d = b · c
a · d
b · d
=
a
b
·
d
d
=
a
b
· 1 =
a
b
a
−b
=
−a
b
= −
a
b
a
b
+
c
b
=
a + c
b
a
b
+
c
d
=
a · d + b · c
b · d
a
b
·
c
d
=
a · c
b · d
a
b
÷
c
d
=
a
b
·
d
c
=
a · d
b · c
a
b
b = a
(4.3.88)
where:
a
b
=
c
d
⇔ ad = bc (4.3.89)
is what we we name a terms ampliﬁcation and:
a
b
+
c
b
=
a + c
b
(4.3.90)
is an operation consisting by putting everything with a common denominator.
We also have the following properties:
P1. The division of several numbers depend on the order of terms. We then say that the
division is a non-commutative operation. This means we have when a that is different
from b and that both are different from zero:
a
b
=
b
a
(4.3.91)
P2. The result of the division of several numbers change if we replace two or more of them by
their intermediate result. We then say that the division is a non-associative operation:
a
b
c
=
a
b
c
(4.3.92)

Draft
P3. The unit is the neutral element has that we multiply the divident or the divider by 1 the
result of the division remains the same.
a
b
=
1 · a
b
=
a
1 · b
= 1 ·
a
b
(4.3.93)
P4. The division may include a divider in such a way that the division is equal to unity (neutral
element 1). We then say that there exist a symmetrical to the division that is obviously
equal to the numerator (dividend) itself.
P5. The incrementation of numerator and denominator by a constant value is not equal to the
initial ratio in the general case where a = b:
a
b
=
a + cte
b + cte
(4.3.94)
Now that we know the multiplication (and therefore power notation) and division, if we consider
a and b are two positive real numbers, different from zero we have:
ap
aq
= a · a . . . · a
p-times
· a · a . . . · a
q-times
= ap+q
(4.3.95)
and:
ap
aq
=
a · a . . . · a
p-times
a · a . . . · a
q-times
= a · a . . . · a
p−q-times
= ap−q
⇒ a−q
=
1
aq
(4.3.96)
and:
ap
· bp
= (a · b)p
⇔
ap
bp
=
a
b
p
(4.3.97)
We have also obviously:
an
an
=
a · a . . . · a
n-times
a · a . . . · a
n-times
= a · a . . . · a
n−n-times
= a0
= 1 (4.3.98)
Also:
(an
)m
= a · a . . . · a
n-times
· . . . a · a . . . · a
n-times
m -times
= a · a . . . · a
mn-times
= amn
(4.3.99)
4.3.2.4.1 n-root
Now that we have introduce in a simple and not too much formal way the operations of
multiplication (and power notation) and division we can introduction the concept of n-root.

Draft
As we know for example that:
23
22
= 23+2
we can by reverse inference for example also write:
21
= 20.5+0.5
= 20.5
20.5
= 21/2
21/2
(4.3.100)
and therefore it means that fractional power exist! This is what we name n-root (in the above
example we speak of 2-root).
We can now deﬁne the principal n-root of any number!
Deﬁnition: In mathematics, the nth root of a number a, where n is a positive integer, is a
number r which, when raised to the power n yields x. That is to say such that: rn
= x, where
n is the degree of the root. By convention we write:
r = x1/n
:= n
√
x (4.3.101)
Roots are usually written using the radical symbol n
√
. . . or also named the radix. The
number n ∈ N is named the radicand and sometimes the index. From what has been said
for the powers, we can easily conclude that the n-th root of a product of several factors is the
product of n-th roots of each factor:
n
√
a ·
n
√
b =
n
√
a · b ⇔
n
√
a
n
√
b
= n
a
b
(4.3.102)
as (seen previously):
(ap
)q
= ap·q
= (aq
)p
(4.3.103)
And therefore:
a
p
q = q
√
ap = ( q
√
a)p
and
p q
√
a = pq
√
a (4.3.104)
Obviously it comes:
n
√
a · n
√
a · . . . n
√
a
n-times
= n (4.3.105)
We also have if a 0:
n
√
an = a (4.3.106)
if n ∈ N∗
is odd and:
n
√
an = |a| (4.3.107)
if n ∈ N∗
is even.
If x 0 and n ∈ N∗
is odd then:
y = a1/n
= n
√
x (4.3.108)

Draft
is the number y such that:
bn
= x (4.3.109)
If n ∈ N∗
is even then obviously, as we already have seen it earlier, the root belong to C (see
section Numbers).
If the denominator of a fraction contains a factor of the form
n
√
ak with a = 0, by multiplying
the numerator and denominator by n
√
an−, we will remove the root of the denominator, since:
x
n
√
ak
n
√
an−k
n
√
an−k
=
x
n
√
an−k
n
√
ak n
√
an−k
=
x
n
√
an−k
n
√
akan−k
=
x
n
√
an−k
n
√
ak+n−k
=
x
n
√
an−k
n
√
an
=
x
n
√
an−k
|a|
(4.3.110)
Example:
Let us see a world famous example of the application of the root about the origin
of the ISO paper formats: A6, A5, A4, A3, A2, A1, A0, etc.
This format of paper has in fact the property (there is a goal at the origin!) to keep the
proportions when we bend or cut the sheet in half in its largest dimension. Thus, if we
denote by L the length and W the width of the sheet, we have:
L
W
=
W
L
2
⇒ L2
= 2l2
(4.3.111)
Hence we have:
L =
√
2W (4.3.112)
As the A0 format by deﬁnition has an area of 1 [m2
]. For this format we have then:
LW = 1 [m2
] (4.3.113)
Therefore we deduce that:
LW =
√
2W2
=
L2
√
2
= 1 [m2
] (4.3.114)
and therefore:
W = 2−1/4
· 1 [m] ∼= 84.1 [cm] (4.3.115)
from whence we derive:
L = 2−1/4
· 1 [m] ∼= 118.9 [cm] (4.3.116)

Draft
4.3.3 Arithmetic Polynomials
Definition: An arithmetic polynomial (not to be confused with algebraic polynomial that
will be studied later in the section Algebra) is a set of numbers separated from each other by
the operators of addition or subtraction (+ or −) including therefore the multiplication...
The components enclosed in the polynomial are known as terms of the polynomial. When
the polynomial contains a single term, then we speak of monomial, if there are two terms we
speak of binomial, and so on...
Theorem 4.7. The value of an arithmetic polynomial is equal to the excess of the sum of the
terms preceded by the + sign on the sum of the terms preceded by the sign −.
Proof 4.7.1.
n1 − n2 + n3 − n4 + n5 − n6 + . . . − ni−1 + ni =
(n1 + n3 + n5 + . . . + ni)
+ (−1)(n2 + n4 + n6 + . . . + ni−1)
(4.3.117)
whatever the values of the terms.
Q.E.D.
Highlight the negative unit −1 is what we name a factorization. The reverse operation is
named a distribution or development.
The product of several polynomials can always be replaced by a single polynomial that we name
the... resulting product. We usually operate as follows: we multiply successively all the terms
of the first polynomial, starting from the left, with the first, the second, ..., the last by the second
polynomial. We obtain a first partial product. We do, if necessary, a reduction (simplification)
of similar terms. We then multiply each of the terms of the partial product successively by the
first, the second, ..., the last term of the third polynomial starting from the left and so on.

Draft
Example:
P1 · P2 · P3 = (a + b + c)(d + e + f)(g + h + i)
= a(d + e + f)(g + h + f) + b(d + e + f)(g + h + i)
+ c(d + e + f)(g + h + i)
= ad(g + h + i) + ae(g + h + i) + af(g + h + i) + bd(g + h + i)
+ be(g + h + i) + bf(g + h + i) + cd(g + h + i) + ce(g + h + i)
+ cf(g + h + i)
= adg + adh + adi + aeg + aeh + aei + afg + afh + afi + bdg + bdh
+ bdi + beg + beh + bei + bfg + bfh + bfi + cdg + cdh + cdi + ceg + ceh
+ cei + cfg + cfh + cfi
(4.3.118)
The product of the polynomials P1, P2, P3, ..., Pk, ... is the sum of all products of ni factors
formed with a term of Pi, of a term of P2, ..., and a term of Pk and so. if there is no reduction,
the number of terms is equal to the product of the numbers of terms of each polynomial such
that the ﬁnal number of therms is equals to:
n =
n
i=1
ni (4.3.119)
4.3.4 Absolute Value
Deﬁnition: In mathematics, the absolute value |x| of a real number x is the non-negative
value of x without regard to its sign. Namely, |x| = x for a positive x, |x| = −x for a negative
x (in which case −x is positive), and |0| = 0. For example, the absolute value of 3 is 3, and the
absolute value of −3 is also 3. The absolute value of a number may be thought of as its distance
from zero.
Remarks
R1. The term absolute value has been used in this sense from at least 1806 in French[3]
and 1857 in English.[4] The notation |x|, with a vertical bar on each side, was introduced
by Karl Weierstrass in 1841.
R2. For plots about the absolute value the reader is referred to the Functional Analysis
section of this book.

Draft
For any real number x, the absolute value x, is formally given by:
|x| =



+ x if x 0
− x if x 0
0 if x = 0
(4.3.120)
At the origin the absolute value was deﬁned as:
|x| =
√
x2 (4.3.121)
We notice that also the following possible notation:
|x| = max(−x, x) (4.3.122)
And the equivalent expressions:
x |x| (4.3.123)
| − x| = |x| (4.3.124)
and also:
|x| y ⇔ −y x y (4.3.125)
|x| y ⇔ x −y ∨ x y (4.3.126)
the latter being often used in the context of solving inequalities.
Example:
Solving an inequality such that:
|x − 3| ≤ 9 (4.3.127)
is then solved simply by using the intuitive concept of distance. The solution is the set of
real numbers whose distance from the real number 3 is less than or equal to 9. This is the
range of center 3 and radius 9 or formally:
[3 − 9, 3 + 9] = [−6, 12] (4.3.128)
Let u indicate that it is also useful to interpret the term:
|x − y| = (x − y)2
(4.3.129)
as the (euclidean!) distance between the two numbers x and y on the real line. Thus, by
providing the set of real numbers of the absolute value distance , it becomes a metric space (see
the section of Topology to have a robust introduction to what is a distance)!!!
The absolute value has some trivial properties that we will give without proof (excepted on
reader request) as they seem to us quite intuitive:
The absolute value has the following four fundamental properties:

Draft
P1. Non-negativity:
|x| ≥ 0 (4.3.130)
P2. Positive-definiteness:
|x| = 0 ⇐⇒ x = 0 (4.3.131)
P3. Multiplicativeness:
|xy| = |x||y| (4.3.132)
P4. Subadditivity (first triangle inequality):
|x + y| ≤ |x| + |y| (4.3.133)
Other important properties of the absolute value include:
P5. Idempotence (the absolute value of the absolute value is the absolute value):
|(|x|)| = |x| (4.3.134)
P6. Evenness (reflection symmetry of the graph):
| − x| = |x| (4.3.135)
P7. Preservation of division (equivalent to multiplicativeness) if y = 0:
x
y
=
|x|
|y|
(4.3.136)
P8. Reverse (second) triangle inequality (equivalent to subadditivity):
|x − y| ≥ |(|x| − y|)| (4.3.137)

Draft
4.3.5 Calculation Rules (operators priorities)
Frequently in computing (in development in particular), we speak of operators precedence.
In mathematics we speak of priority of the sets of operations and rules of signs. What is this
exactly?
We have already seen what are the properties of addition, subtraction, multiplication, division
and power. We therefore insist that the reader distinguishes the concept of property of this of
priority (that we will immediately see) which are (obviously) two completely different things!
In mathematics, in particular, we first define the priorities of the symbols {[()]}:
1. Operations that are in brackets () should be performed first in the polynomial.
2. Operations that are in brackets [] should be made afterwards from the results of operations
that were in brackets ().
3. Finally, from the intermediate results of operations that were in () and brackets [], we
calculate the operations that are between the braces {}.
Let us do an example, this will be more telling.
Example:
Consider the calculation of the polynomial:
{[5 · (8 + 2) + 3 · [4 + (8 + 6) · 2]] · (1 + 9)} · 7 + 1 (4.3.138)
According to the rules we defined earlier, we first calculate all the elements that are in
parenthesis (), that is to say:
8 + 2 = 10 (8 + 6) = 14 (1 + 9) = 10 (4.3.139)
Which give us:
{[5 · 10 + 3 · [4 + 14 · 2]] · 10} · 7 + 1 (4.3.140)
Always according to the rules we defined earlier, now we calculate all the elements
between brackets by always starting to calculate the terms that are in brackets [] at the
lowest level of the other brackets []. Thus, we first calculate the expression [4 + 14 · 2]
that is in the top-level bracket: [5 · 10 + 3 · ...].
This give us [4 + 14 · 2] = 32 and therefore:
{[5 · 10 + 3 · 32] · 10} · 7 + 1 (4.3.141)

Draft
It remains to us to calculate now [5 · 10 + 3 · 32] = 146 and therefore:
{146 · 10} · 7 + 1 (4.3.142)
We now calculate the single term in braces, which gives us:
{146 · 10} = 1460 (4.3.143)
Finally it remains:
1460 · 7 + 1 = 10221 (4.3.144)
Obviously this is a special case ... But the idea remains the same in general.
The priority of arithmetic operators is a problem mainly related to computer languages (as we
have already mentioned) because we can only write mathematical relation on a single line and
this is many times as source of confusion for people not having technical skill.
−a · (b + c)d
ef
− g (4.3.145)
will be written (pretty much on most computer languages):
−a ∗ (b + c)∧
d/e∧
f − g (4.3.146)
A non initiated could read this in many ways:
−a · (b + c)d
ef
− g;
[−a · (b + c)]d
ef
− g;
−a · (b + c)d
e
f
− g; [−a · (b + c)]
d
ef
− g
(4.3.147)
Thus it has logically be defined an order of prioritization of operators such that the operations
are carried out in the following order:
1. − Negation
2. ˆ Power
3. ∗ Multiplication and / division
4. Integer division (specific to computer science)
5. mod Module (see section Number Theory)
6. +, − Addition and subtraction
Obviously the rules of parentheses (), brackets [], and braces {} that were defined in mathemat-
ics apply also to computing.
Thus we get in the order (we replace every transaction made with a symbol):

Draft
First the terms in parentheses:
−a ∗ (b + c)∧
d/e∧
f − g = −a ∗ α∧
d/e∧
f − g (4.3.148)
1. First the negation (rule 1):
−a ∗ α∧
d/e∧
f − g = β ∗ α∧
d/e∧
f − g (4.3.149)
2. The power (rule 2):
β ∗ α∧
d/e∧
f − g = β ∗ χ/δ − g (4.3.150)
3. We apply the multiplication (rule 3):
β ∗ χ/δ − g = ε/δ − g (4.3.151)
4. And we apply division (rule 3 again):
ε/δ − g = φ − g (4.3.152)
The rules (4) and (5) does not apply to this particular example.
5. And Finally (rule 6):
φ − g = ϕ (4.3.153)
Thus, following these rules, neither a computer nor ahuman can (should) be wrong in interpret-
ing an equation written on a single line.
In computer code, however, there are several operators that we do not always ﬁnd in pure
mathematics and which order property frequently change depending from a computer language
to another. We will not dwell too much on that stuff as it is almost without end, however, we
have below a small description:
• The concatenation operator is evaluated before comparisons operators.
• Comparison operators (=, , , ...) all have equal priority.
However, the leftmost operator in an expression, hold a higher priority.
The logical operators are evaluated in the following order of priority in most computing lan-
guages:
1. Not (¬)
2. And (∧)
3. Or (∨)

Draft
4. Xor (⊕)
5. Eqv (⇔)
6. Imp (⇒)
Now that we have seen the operator priorities, what are the rules about signs applicable in
mathematics and computing science?
First, you must know that these latter rules only apply in the case of multiplication and division.
Given two positive numbers (+x), (+y). We have:
(+x) · (+y) = (+x) · (+y) = +(x · y) (4.3.154)
In other words, the multiplication of two positive numbers is a positive number and this can be
generalized to the multiplication of n positive numbers.
We have:
(−x) · (+y) = (+x) · (−y) = −(x · y) (4.3.155)
In other words, the multiplication of a positive number to a negative number is negative. Which
can be generalized: to a positive result of a multiplication if there is an even number of negative
numbers, and a negative result if there is an odd number of negative numbers on all n numbers
included in the multiplication.
We have:
(−x) · (−y) = (−x) · (−y) = +(x · y) (4.3.156)
In other words, multiplying two negative numbers is positive. What can be generalized: to
a positive result of the multiplication if there is an even number of negative numbers and a
negative result if there is an odd number of negative numbers.
About divisions, the reasoning is the same:
(+x)
(+y)
= +
x
y
and
(+y)
(+x)
= +
y
x
(4.3.157)
In other words, if the numerator and denominator are positive, then the result of the division
will be positive.
We have:
(+x)
(−y)
=
(−x)
(+y)
= −
x
y
and
(−y)
(+x)
=
(+y)
(−x)
= −
y
x
(4.3.158)
In other words, if either the numerator or denominator is negative, then the result of the division
will be necessarily negative.
We have:
(−x)
(−y)
= +
x
y
and
(−y)
(−x)
= +
y
x
(4.3.159)

Draft
In other words, if the numerator and denominator are positive, then the result of the division,
will necessarily be positive.
Obviously if we have a subtraction of terms, it is possible to rewrite it in the form:
x − y = x + (−1)y = −1 · (−x + y) (4.3.160)
∼80 %
11 votes, 76.36%

Draft
4.4 Number Theory
T
Raditionally, number theory is a branch of mathematics that deals with properties of
integers, whether natural or whole integers. More generally, the ﬁeld of study of
this theory concerns a broad class of problems that naturally come from the study
of integers. Number theory can be divided into several branches of study (algebraic
number theory, computational number theory, etc.) depending on the methods used and the
issues addressed.
Remark
The sign of the cross × for multiplication is said to be for the ﬁrst time in the book
of Oughtred (1631), about the halfway point (modern notation for multiplication), we
ought it to Leibniz. From 1544, Stiefel, in one of his books did not employ any sign and
designated the product of two numbers by placing them next to each other.
We chose to introduce in this section only the subjects that are essential to the study of mathe-
matics and theoretical physics of this book as well as those to be absolutely part of the general
culture of the engineer (some results have application in Biostatistics!).
4.4.1 Principle of good order
We will take for granted the principle that says that every nonempty set S ⊂ N contains a
smaller element.
We can use this theorem to prove an important property of numbers named Archimedean
property or Archimedes’ axiom which states:
For ∀a, b ∈ N where a is non-zero, there is at least one positive integer n such that:
n · a b (4.4.1)
In other words, for two unequal values, there is always an integer multiple of the smallest, bigger
than the larger one. We name Archimedean structures whose elements satisfy a comparison
property (see section Set Theory).
While this is trivial to understand in the case of integers let us prove it because it allows us to see
the type of approaches used by mathematicians when they must prove trivial items like this...
Proof 4.7.2. Let us suppose the opposite by saying that for ∀n ∈ N we have:
n · a b (4.4.2)
If we can prove that it is absurd for any n then we will have prove the Archimedean property
(and also if a, b are real).
Let us consider then the set:
S = {b − na|n ∈ N} (4.4.3)

Draft
Using the principle of good order, we deduce that there exist s0 ∈ S such as s0 ≤ s for all
s ∈ S. Let us write that this smaller element is:
s0 = b − n0a (4.4.4)
and therefore we also have:
b − (n0 + 1)a ∈ S (4.4.5)
As by hypothesis na ≤ b then we must have:
b − (n0 + 1)a ≥ b − n0a (4.4.6)
and if we reorganize and simplify:
−(n0 + 1) ≥ −n0 (4.4.7)
and that we simplify the negative sign we had to get...:
n0 + 1 ≥ n0 (4.4.8)
hence an obvious contradiction!
This contradiction leads that the initial assumption as na b for all n then is false and therefore
the Archimedean property is proved by the absurd.
Q.E.D.
4.4.2 Induction Principle
Let S be a set of natural numbers that has the following two properties:
P1. 1 ∈ S
P2. If k ∈ S, then k + 1 ∈ S
then:
S = N {0} = N∗
(4.4.9)
We are build like this the set of natural numbers (refer to the section Set Theory to see the
rigorous construction of the set of naturla number with the Zermelo-Fraenkel axioms).
Theorem 4.8. Given now:
B = N∗
S (4.4.10)
the symbol meaning for recall excluding. We want to prove that:
B = ∅ (4.4.11)

Draft
Again, even if it is trivial to understand, let us do the proof because it allows us to see the type
of approaches used by mathematicians when they must prove trivial stuff like this...
Proof 4.8.1. Let us suppose the opposite, that is to say:
B = ∅ (4.4.12)
By the principle of good order, since B ⊂ N, B must have a smallest element which we will
denote by b0.
But since 1 ∈ S by the property (P1), we have that b0 1 and of course also that 1 ∈ B, that is
to say also b0 − 1 ∈ S. By using the property (P2), we finally have that b0 ∈ S, that is to say
that b0 ∈ B, therefore we get a contradiction.
Q.E.D.
Example:
We want to show thanks to the induction principle, that the sum of the first n
square equals n(n+1)(2n +1)/6, that is to say for n ≥ 1, we would have to (see section
Sequences and Series):
1 + 2 + . . . + n2
=
n
i=1
i2
=
n(n + 1)(2n + 1)
6
(4.4.13)
First the above relation is easily verified for n = 1 we will show that n = k + 1 also
verifies that relation. Under the induction hypothesis:
12
+ 22
+ . . . + k2
+ (k + 1)2
=
k+1
i=1
i2
=
k(k + 1)(2k + 1)
6
+ (k + 1)2
=
(k + 1)(k + 2)(2k + 3)
6
(4.4.14)
although we fall back on the assumption of the validity of the first relation but with
n = k + 1, hence the result.
This prove process is therefore of great importance in the study of arithmetic. Often observation
and induction have led to a suspicion of laws it would have been more difficult to find by a priori.
We realize the accuracy of formulas by the previous method that gave birth to modern algebra
by Fermat and Pascal studies on the Pascal’s triangle (see section Calculus).

Draft
4.4.3 Divisibility
Deﬁnition: Given A, B ∈ Z with A = 0. We say that A divides B (without rest) if there is
an integer q (the quotient) such that:
B = Aq (4.4.15)
in which case we write to differentiate of the class division:
A|B (4.4.16)
Otherwise, we write
A B (4.4.17)
and we say that A does not divide B.
Remarks
R1. Remember that | is a relation when the symbol / is an operation!
R2. Do not confuse the expression A divides B which means that rest is necessarily
zero and the expression A is the divisor of the division by B indicating that the rest is
not necessarily zero!
Moreover, if A|B, we also say that B can be divided by A or B is a multiple of A.
In case where A|B and that 1 ≥ A B, we will say that A is a proper divisor of B.
Moreover, it is clear that A|0 regardless of A ∈ Z {0} otherwise what we have a singularity.
Here are some basic theorems relating to the division:
Theorem 4.9. If A|B, then A|BC whatever C ∈ Z. Or more formally:
∀C ∈ Z : A|B ⇒ A|BC (4.4.18)
Proof 4.9.1. If A|B, the it exists an integer q such that:
B = Aq (4.4.19)
Then:
BC = (Aq)C = A(qC) (4.4.20)
and therefore:
A|BC (4.4.21)
Q.E.D.

Draft
Theorem 4.10. If A|B and B|C, then A|C or more formally:
A|B ∧ A|B ⇒ A|C (4.4.22)
Proof 4.10.1. If A|B and B|C then, there exists two integers q and r such that B = Aq and
C = Br. More formally:
A|B ∧ B|C → ∃(q, r) ∈ N : B = Aq ∧ C = Br (4.4.23)
Therefore:
C = A(qr) (4.4.24)
and hence:
C = A|C (4.4.25)
Q.E.D.
Theorem 4.11. If A|B and A|C then:
A|(Bx + Cy) ∀x, y ∈ Z (4.4.26)
Proof 4.11.1. If A|B and A|C then, there exists two integers q and r such that B = Aq and
C = Ar. It follows:
Bx + Cy = (Aq)x + (Ar)y = A(qx + ry) (4.4.27)
and therefore:
A|(Bx + Cy) ∀x, y ∈ Z (4.4.28)
Q.E.D.
Theorem 4.12. If A|B and B|A then:
A = ±B (4.4.29)
Proof 4.12.1. If A|B and B|A then, there exists two integers q and r such that B = Aq and
A = Br.
We then have:
B = B(qr) (4.4.30)
and thus qr = 1. This is why we can have q = ±1 if r = ±1 and thus:
A = ±B (4.4.31)
Q.E.D.
Theorem 4.13. If A|B and B == the:
|A| ≤ |B| (4.4.32)
Proof 4.13.1. If A|B then there exist an integer q = 0 such that B = Aq. But then:
|B| = |A||q| ≥ |A| (4.4.33)
as |q| ≥ 1.
Q.E.D.

Draft
4.4.3.1 Euclidean Division
The Euclidean division is an operation that, to two integers named respectively the dividend
and divisor combines two other integers named the quotient and remainder. Initially de-
ﬁne only for nonzero integers, it can be generalized to relative integers and polynomials, for
example.
Deﬁnition: We name euclidean division or integer division of two numbers A and B the
operation of dividing B by A, stopping when the rest is strictly less than A.
Let us recall (see section Numbers) that any number which admits exactly two euclidean divi-
sors (such that division gives no remainder) that are the 1 and itself is named a prime number
(which excludes the number 1 of the list of primes) and that any pair of numbers which have
only 1 as common Euclidean divider are say to be relatively prime, mutually prime, or
coprime.
Theorem 4.14. Given A, B ∈ Z with A 0. The theorem of the Euclidean division state
that there are unique integers q (quotient) and r (remainder) such as:
B = Aq + r ⇔ r = B − Aq (4.4.34)
where 0 ≥ r A. Furthermore, if A B, then 0 r A.
Example:
One cake with 9 parts (B), we then have to divide it between 4 people (A) with
one part remaining (r=1) such that q=2
Figure 4.19 – The pie has 9 slices, so each of the 4 people receive 2 slices and 1 is left over.
and therefore:
10 = 2 · 4 + 1 (4.4.35)
Proof 4.14.1. Let us consider the set:
S = {r = B − qA|q, B ∈ Z, A ∈ Z∗
, B − qA ≥ 0} (4.4.36)

Draft
It is relatively easy to see that S ⊂ N∗
{0} and that S = ∅, hence, according to the principle of
good order, we conclude that S contains a smaller element r ≥ 0. Given q the integer satisfying
thus:
r = B − Aq (4.4.37)
We want to ﬁrst show that r A assuming the opposite (proof ad absurdum), that is to say that
r = A. So, in this case, we have:
B − qA = r ≥ A (4.4.38)
which is equivalent to:
B − (q + 1)A = r − A ≥ 0 (4.4.39)
but B − (q + 1)A ∈ S and:
B − (q + 1)A B − qA (4.4.40)
This contradicts the fact that:
r = B − qA (4.4.41)
is the smallest element of S. So r A. Finally, it is clear that if r = 0, we have A|B, hence the
second statement of the theorem.
Q.E.D.
Remark
In the statement of the Euclidean division, we assumed that A 0. What do we get when
A 0? In this situation, −A is obviously positive, and then we can apply the Euclidean
division to B and −A. Therefore, there are integers q and r integers such that:
B = q(−A) + r (4.4.42)
where 0 ≥ r |A|. But this relation can be written:
B = −q(A) + R (4.4.43)
where obviously, −q is an integer. The conclusion is that the Euclidean division can be
stated in a more general form.
Given A, B ∈ Z, there exist two integers q and r such that:
B = Aq + r (4.4.44)
where 0 ≥ r |A|. Furthermore, if A B, then 0 r |A|
The integers q and r are unique in the Euclidean division. Indeed, if there are two other integers
q and r such as:
B = Aq + r (4.4.45)

Draft
always with 0 ≤ r A, then:
A(q − q) = r − r (4.4.46)
and therefore:
A|(r − r ) (4.4.47)
Following theorem 4.13 we have if r − r = 0 that |r − r | ≥ A.
But, this last inequality is impossible as by construction −A r − r . Therefore r = r and, as
A = 0, then q = q hence the unicity.
4.4.3.1.1 Greatest common divisor
The greatest common divisor (gcd) (also known as greatest common factor (gcf), high-
est common factor (hcf), greatest common measure (gcm), or highest common divisor) of
two or more integers, when at least one of them is not zero, is the largest positive integer that
divides the numbers without a remainder.
Deﬁnition: Given a, b ∈ Z such as ab = 0. The greatest common divisor (gc) of a and b,
denoted:
(a, b) (4.4.48)
is the positive integer n that satisﬁes the following two properties:
P1. d|a and d|b (so without remainder r in the division!)
P2. If c|a and c|b the c ≤ d and c|d (by division!)
Note that 1 is always a common divisor of two arbitrary integers.

Draft
Example:
Let us consider the positive integers 36 and 54. A common divisor of 36 and 54
is a positive integer that divides 36, and also 54. For example, 1 and 2 are common
divisors 36 and 54.
Div36 = {1, 2, 3, 4, 6, 9, 12, 18, 36}
Div54 = {1, 2, 3, 6, 9, 18, 27, 54}
(4.4.49)
We have the intersection represented by the following Venn diagram:
Figure 4.20 – Venn diagram of common divisors
with the following set of common divisors:
However it is not necessarily obvious that the greatest common divisor other than 1 (that is to
say different of 1) of two integers a and b that are not relatively prime always exists. This is
proved by the following theorem (however, if the gcd exists, it is by deﬁnition unique!) named
Bézout theorem that can also gives the opportunity to prove other interesting properties of
two numbers as we shall see later .
Theorem 4.15. Given a, b ∈ Zsuch that ab = 0. If d divides a and d divides b (for both without
remainder r !) then there must two integers x and y such that:
d = (a, b) = ax + by (4.4.50)
This relationship is named the Bézout identity and it is a linear Diophantine equation (see
section Calculus).
Proof 4.15.1. Obviously, if a and b are relatively prime we know that d is then 1.
To prove the Bézout identity let ﬁrst consider the set:
S = {d = ax + by|x, y ∈ Z, ax + by 0} (4.4.51)
As S ⊂ N and S == ∅, we can use the principle of good order and conclude that S has a
smaller element d. We can then write:
d = ax0 + by0 (4.4.52)

Draft
for some given choice x0, y0 ∈ Z. So it is sufficient to prove that d = (a, b) to prove the Bézout
identity!
Let us proceed with a proof by contradiction by assuming d a. Then if this is the case,
following the Euclidean division, there exist q, r ∈ Z such as a = qd + r, where 0 r d.
But then:
r = a − qd = a − q(ax0 + by0) = a(1 − qx0) + b(−qy0) (4.4.53)
Thus we have that r ∈ S and r d, which contradicts the fact that d is the smallest possible
element of S. Thus we have proven not only that d|a, but also that d always exists and, in the
same way we prove that d|b.
As important corollary let us now prove that if a, b ∈ Z such that ab = 0, then:
S = {ax + by|x, y ∈ Z} (4.4.54)
is the set of all multiples of d(a, b): As d|a and d|b, then we have necessarily dax + by| for any
x, y ∈ Z. Either M = {nd|n ∈ Z}. Our problem is then reduced to prove the fact that S = M.
Given first s ∈ S which means that d|s and involves s ∈ M.
Given a m ∈ M, this would mean that m = nd for a certain n ∈ Z.
As d = ax0 + by0 for any choice of integers x0, y0 ∈ Z, then:
m = nd = n(ax0 + by0) = a(nx0) + b(ny0) ∈ S (4.4.55)
Q.E.D.
The assumptions can seem complicated but put your attention a given time on the last equality.
You will quickly understand!
Remark
If instead of defining the greatest common divisor of two non-zero integers, we allow one
of them to be equal to 0, say: a = b, b = 0. In this case, we have a|b and, according to
our definition of the GCD, it is clear that (a, 0) = |a|.
Given d = (a, b) and m ∈ Z, then we have the following properties of the GCD (without proof
but if a reader request them we will give the details):
P1. (a, b + ma) = (a, b) = (a, −b)
P2. (am, bm) = |m|(a, b) where m = 0
P3.
a
d
,
b
d
= 1

Draft
P4. If g ∈ Z {0} such that g|a and g|b then
a
g
,
b
g
=
1
|g|
(a, b)
In some books, these four properties are proved using intrinsically the property itself. Personally
we abstain make usage of this approach because doing this is more ridiculous than anything else
as the statement of the property is a proof in itself.
Let us now develop a method (algorithm) that will be very useful to us to calculate (determine)
the greatest common divisor of two integers (sometimes useful computing science).
4.4.3.2 Euclidean Algorithm
The Euclidean algorithm is an algorithm for determining the greatest common divisor of two
integers (we have hesitate to put this subject in the section of Numerical Methods...).
To address this method intuitively, you must know that that you need to see that an integer as
a length, a pair of integers as a rectangle (sides) and their GCD is the size of the largest square
for tile (paving) their rectangle by deﬁnition (yes if you think for a moment it’s quite logical!).
The algorithm decomposes the original rectangle into squares, always smaller and smaller, by
successive Euclidean division of the length by the width, then the width by the remainder until a
zero remainder. We must understand this geometric approach to then understand the algorithm.
Example:
Let us consider that we seek the GCD of (a, b) where b is equal 21 and a is equal
15 and keep in mind that the GCD, besides the fact that it divides a and b, must leave a
zero remainder! In other words it must divide the remainder of the division of b by a also!
So we have the following rectangle of 21 by 15:
Figure 4.21 – First step of the GCD algorithm

Draft
First we see if 15 is the GCD (it always starts with the smallest). We then divide 21 by
15, which is equivalent geometrically to:
Figure 4.22 – Second step of the GCD algorithm
15 is therefore not the GCD (we suspected it...). We immediately see that we can not
pave the rectangle with a square of 15 by 15.
So we have a remainder of 6 (left rectangle). The GCD as we know must, if it exists, by
deﬁnition divide that remains and leave a zero remainder.
So we have a rectangle of 15 by 6. So we are looking now to pave this new rectangle
because we know that the greatest common divisor is by construction less than or equal
to 6. Then we have:
Figure 4.23 – Third step of the GCD algorithm
So we divide 15 by remainder 6 (this result will be less than 6 and immediately permits
to tests whether the reamainder will be the GCD). We are getting:
Figure 4.24 – Fourth step of the GCD algorithm
Again, we can not pave the rectangle only with squares. In other words, we have a
non-zero remainder which is 3. Given now a rectangle of 6 by 3. So we are looking
now to pave the new rectangle because we know that the greatest common divisor is by
construction less than or equal to 3 and that it will leave a remainder equal to zero, if it
exists. We then have geometrically:

Draft
Figure 4.25 – Fifth step of the GCD algorithm
We divide 6 by 3 (which will be less than 3 and permits us to test immediately whether
the rest will be the GCD):
Figure 4.26 – Sixth step of the GCD algorithm
and it’s all good! We then have 3 that leave us with a remainder equal to zero and divides
the remainder 6 so this is the GCD. So we have in the end:
Figure 4.27 – Summary of the GCD algorithm
Now let us see the equivalent formal approach.
Given a, b ∈ Z, where a 0. Applying successively the Euclidean division (with b a), we
get the following sequence of equations:
b = qa1 + r1 0 r1 a
a = r1q2 + r2 0 r2 r1
r1 = r12q3 + r3 0 r3 r2
. . .
rj−2 = rj−1qj + rj 0 rj rj−1
rj−1 = rjqj+1
(4.4.56)
if d = (a, b), then d = rj. with the corresponding pseudo-code algorithm:

Draft
Algorithm 1: Proportional Parts pseudo-algorithm
Data: a,b
Result: b
1 initialization;
2 r = a mod b;
3 while r = 0 do
4 a := b;
5 b : r;
6 f(b) := f(a);
7 a := x1;
8 f(a) := f(x1);
9 display x1;
Otherwise even more formally:
Proof 4.15.2. We want ﬁrst prove that rj = (a, b). But, following the property P1:
(a, b + ma) = (a, b) = (a, −b) (4.4.57)
we have:
(a, b) = (a, r1) = (r1, r2) = . . . = (rj−1, rj) (4.4.58)
To prove the second property of the Euclide’s algorithm, we write the prior-previous equation
of the system under the form:
rj = rj−2 − qjrj−1 (4.4.59)
Now, using the previous equation this prior-previous equation of the system, we have:
rj = rj−2 − qj(rj−3 − qj−1rj−2) = (1 + qjqj−1)rj−2 + (−qj)rj−3 (4.4.60)
Continuing this process, we can express rj as a linear combination of a and b.
Q.E.D.

Draft
Example:
Let us calculate the greatest common divisor of (429, 966) and express this num-
ber as a linear combination of 429 and 966.
966 = 429 · 2 + 108
429 = 108 · 3 + 105
108 = 105 · 1 + 3
105 = 35 · 3
(4.4.61)
We therefore conclude that:
rj = d = (966, 429) = 3 (4.4.62)
and, in addition, that:
3 = 108 − 105 · 1 = 108 − (429 − 108 · 3) = 108 · 4 − 429
= (966 − 429 · 2) · 4 − 429
= 966 · 4 − 429 · 9 = 966 · 4 + 429 · (−9)
(4.4.63)
Thus the GCD is indeed expressed as a linear combination of a and b and constitutes as
such the GCD.
Definition: We say that the integers a1, a2, . . . , an are relatively prime or if:
(a1, a2, . . . , an) = 1 (4.4.64)
4.4.3.3 Least Common Multiple
The least common multiple (also named the lowest common multiple or smallest common
multiple) of two integers a and b, usually denoted by LCM(a, b), is the smallest positive integer
that is divisible by both a and b. Since division of integers by zero is undefined, this definition
has meaning only if a and b are both different from zero.
The LCM is familiar from grade-school arithmetic as the lowest common denominator (LCD)
that must be determined before fractions can be added, subtracted or compared. The LCM of
more than two integers is also well-defined: it is the smallest positive integer that is divisible by
each of them.
Definitions:
D1. Given a1, a2, . . . , an ∈ Z {0}, we say that m is a common multiple of a1, a2, . . . , an
if ai|m for i = 1, 2, . . . , n
D2. Given a1, a2, . . . , an ∈ Z {0}, we name lowest common multiple (LCM) of
a1, a2, . . . , an if ai|m for i = 1, 2, . . . , n denoted:
[a1, a2, . . . , an] (4.4.65)

Draft
the lowest integer positive common multiple to all common multiples of a1, a2, . . . , an.
Examples:
E1. Let us consider the positive integers 3 and 5. A common multiple of 3 and 5
is a positive integer which is both a multiple of 3, and a multiple of 5. In other words,
which is divisible by 3 and 5. We have therefore:
M3 = {3, 6, 9, 12, 15, 18, 21, 24, 27, 30, . . .}
M5 = {5, 10, 15, 20, 25, 30, 35, 40, 45, . . .}
(4.4.66)
We then have the intersection represented by the following Venn diagram:
with then have following set of common multiples:
M3 ∩ M5 = {15, 30, 45, 60, . . .} (4.4.67)
and therefore the LCM is given by:
LCM = min{15, 30, 45, 60, . . .} = 15 (4.4.68)
Or if it can help here is another possible visualization of the concept:
We see obviously that all the common multiples of 3 and 5 is the set of multiples of 15.

Draft
Examples:
E2. Wikipedia gives also a nice visual example of a Venn Diagram when we seek
for a LCM of 5 multiples using a visual tool:
Remark
Given a1, a2, . . . , an ∈ Z {0}. Then the least common multiple exists. Indeed, consider
the set E of natural integers m that for all i divide ai. What we will write:
E = {m|ai|m ∈ N, i = 1, 2, . . . , n} (4.4.69)
Since we have necessarily |a1a2 . . . an| ∈ E, then the set is not empty and, according to
the axiom of good order, the set E contains a smaller positive element.
Let us now see some theorems related to the LCM:
Theorem 4.16. If m is any common multiple of a1, a2, . . . , an then [a1, a2, . . . , an]|m that is to
say that m divides each of the ai.
Proof 4.16.1. Given M = [a1, a2, . . . , an]. Then, by the Euclidean division, there are integers
q and r such that:
m = qM + r 0 ≤ r M (4.4.70)
It sufﬁces to show that r = 0. Let us suppose that r = 0 (reductio ad absurdum). Since ai|m and
ai|M, the we have ai|r and this for i = 1, 2, . . . , n. So r is common multiple of a1, a2, . . . , anof
the smallest than the LCM. We just obtained a contradiction, which proves the theorem.

Draft
Q.E.D.
Theorem 4.17. If k , then [ka1, ka2, . . . , kan] = k[a1, a2, . . . , an]
The proof will be assumed obvious (if not as always contact us and will add the details!)
Theorem 4.18. [a, b] · (a, b) = |ab|
Proof 4.18.1.
Lemma 4.18.1. For this proof, we will use the Euclid’s lemma that says that if a|bc and
(a, b) = 1 then a|c.
In other words Euclid’s lemma captures a fundamental property of prime numbers, namely: If
a prime divides the product of two numbers, it must divide at least one of those numbers. It
is also named Euclid’s first theorem. This lemma is the key of the proof of the fundamental
theorem of arithmetic that we will see just further below.
Indeed, this can be easily verified because we have seen that there exists x, y ∈ Z such as
1 = ax + by and then c = acx + bcy. But a|ac and a|bc imply that a|(acx + bcy), that is to say
also that a|c.
Ok let us now return to our theorem:
Since (a, b) = (a, −b) and [a, b] = [a, −b], it suffices to prove the result for positive integers a
and b.
First of all, let consider the case where (a, b) = 1. The integer [a, b] being a multiple of a, we
can write [a, b] = ma. Thus, we have b|ma and since (a, b) = 01, it follows, by Euclid’s lemma,
that b|m. Therefore, b ≤ m and then ab ≤ am. But ab is a common multiple of a and b that can
not be smaller than the LCM. therefore ab = ma = [a, b].
For the general case, that is to say (a, b) = d 1, we have, according to the property:
a
d
,
b
d
= 1 (4.4.71)
and with the result obtained previously that:
a
d
,
b
d
a
d
,
b
d
=
a
d
b
d
(4.4.72)
When we multiply both sides of the equation by d2
, the result follows and the proof is done.
Q.E.D.
4.4.3.4 Fundamental Theorem of Arithmetic
The fundamental theorem of arithmetic says that every natural number n 1 can be written as
a product of primes, and this representation is unique, except for the order in which the prime
factors are arranged.

Draft
The theorem establishes the importance of prime numbers. Essentially, they are the building
blocks of building positive integers, each positive integer containing primes in a unique way.
Remark
This theorem is sometimes named factorization theorem (wrongly ... because some
other theorems have the same name ...).
So let’s go:
Theorem 4.19. Every integer greater than 1 either is prime itself or is the product of prime
numbers, and that this product is unique, up to the order of the factors.
Remark
This theorem is one of the main reasons why 1 is not considered a prime number: if 1
were prime, the factorization would not be unique.
Proof 4.19.1. The proof uses Euclid’s lemma: if a prime p divides the product of two natural
numbers a and b, then either p divides a or p divides b (or both).
If n is prime, and therefore product of a unique prime integer, namely itself, the result is true
and the proof is complete (say that a prime number is product of itself is obviously a misnomer!
). Suppose that n is not prime and therefore strictly greater than 1 and consider the set:
D = {d|d|n and 1 d n} (4.4.73)
So, D ⊂ N and since n is composite, we have that D = ∅. According to the principle of
good order, D has a smaller element p1 that is prime, otherwise the minimum choice of p1 is
contradicted. We can the write n = p1n1. If n1 is prime, then the proof is complete. If n1 is also
composite, then we repeat the same argument as before and we deduce the existence of a prime
number p2 and of an integer n2 n1, such as n = p1p2n2. By continuing we come inevitably
to the conclusion that nk will be prime.
So ﬁnally we well show that any number can be decomposed into prime numbers factors with
the principle of good order.
Q.E.D.
{pn ∈ N|pn
√
m} (4.4.74)
a = b mod(m) (4.4.75)

Draft
a = b + km (4.4.76)
m|(a − b) (4.4.77)
c = b + lm = (a + km) + lm = a + (k + l)m (4.4.78)
a = b + km, a = b + lm (4.4.79)
a + a = b + b + (k + l)m (4.4.80)
aa = bb + blm + b km + klm2
= bb + sm (4.4.81)
b − a
r
= k,
b − a
s
= l ⇒ b − a = rk = sl (4.4.82)
f(x) = Axn
+ Bn−1
+ Cxn−2
+ . . . + Kx + L (4.4.83)
{. . . , −9, −6, −3, 0, 3, 6, 9, . . .}
{. . . , −8, −5, −2, 1, 4, 7, 10, . . .}
{. . . , −7, −4, −1, 2, 5, 8, 11, . . .}
(4.4.84)
[0]3 + [0]3 = [0]3; [0]3 + [1]3 = [1]3
[0]3 + [2]3 = [2]3; [1]3 + [1]3 = [2]3
[1]3 + [2]3 = [0]3; [2]3 + [2]3 = [1]3
(4.4.85)

Draft
[0]3 × [0]3 = [0]3; [0]3 × [1]3 = [0]3
[0]3 × [2]3 = [0]3; [1]3 × [1]3 = [1]3
[1]3 × [2]3 = [2]3; [2]3 × [2]3 = [1]3
(4.4.86)
[a]m + [b]m = [a + b]m (4.4.87)
[a]m · [b]m = [a · b]m (4.4.88)
[a]m + [0]m = [a]m, ∀a ∈ Z, ∀m ∈ N (4.4.89)
[a]m · [1]m = [a]m, ∀a ∈ Z, ∀m ∈ N (4.4.90)
a
b
= q1 +
r1
b
b
r1
= q2 +
r2
r1
r1
r2
= q3 +
r3
r2
. . .
rn−2
rn−1
= qn
(4.4.91)
qn =
1
εn−1
(4.4.92)
1
εn−1
= qn + εn (4.4.93)
r = A − a2
= (
√
A − a)(
√
A + a) (4.4.94)

Draft
√
A − a =
r
√
A + a
(4.4.95)
√
A = a +
r
√
A + a
(4.4.96)
√
A = a +
r
2a +
r
√
A + a
(4.4.97)
C1 = [q1] = q1
C2 = [q1, q2] = q1 +
1
q2
=
q1q2 + 1
q2
C3 = [q1, q2, q3] = q1 +
1
q2 +
1
q3
=
(q1q2 + 1) · q3 + q1
q2q3 + 1
C4 = [q1, q2, q3, q4] =
((q1q2 + 1) · q3 + q1) · q4 + (q1q2 + 1)
(q2q3 + 1) · q4 + q2
(4.4.98)
n0 = 1, n1 = q1, . . . , ni = qini−1 + ni−2
d0 = 0, d1 = 1, . . . , di = qidi−1 + di−2
(4.4.99)
0 = d0 d1 d2 d3 . . . (4.4.100)
C1 =
n1
d1
, C2 =
n2
d2
, C3 =
n3
d3
, . . . , Ci =
ni
di
(4.4.101)
Ck = [q1, q2, q3, . . . , qk] =
nk
dk
(4.4.102)
nidi−1 − dini−1 = (−1)i
(4.4.103)

Draft
ni+1di − di+1ni = (qi+1ni + ni−1)di − (qi+1di + di−1)ni = ni−1di − di−1ni = (−1)i+1
(4.4.104)
C1 C3 C5 . . . C6 C4 C2 (4.4.105)
Ck+2 − Ck = (Ck+2 − Ck+1) + (Ck+1 − Ck) =
nk+2
dk+2
−
nk+1
dk+1
+
nk+1
dk+1
−
nk
dk
=
nk+2dk+1 − nk+1dk+2
dk+2dk+1
+
nk+1dk − nkdk+1
dk+1dk
=
(−1)k+2
dk+2dk+1
+
(−1)k+1
dk+1dk
=
(−1)k+2
dk + (−1)k+1
dk+2
dk+2dk+1dk
=
(−1)k+1
(−1)dk + (−1)k+1
dk+2
dk+2dk+1dk
=
(−1)k+1
(dk+2 − dk)
dk+2dk+1dk
(4.4.106)
C1 C3 C5 . . . et C2 C4 C6 . . . (4.4.107)
Ck − Ck−1 =
nk
dk
−
nk−1
dk−1
=
nkdk−1 − nk−1dk
dkdk−1
=
(−1)k
dkdk−1
(4.4.108)
a
b
= q1 +
1
1/ε1
= q1 +
1
q2 + 1/ε2
= . . . (4.4.109)
x = loga(u) =
ln(u)
ln(a)
(4.4.110)
y−1 = u y0 = a yn+1 =
yn−1
yq1
n
(4.4.111)
εn =
ln(yn−2)
ln(yn−1)
− qn (4.4.112)

Draft
ε1 = x − q1 =
ln(u)
ln(a)
− q1 =
ln(yn−2)
ln(yn−1)
− qn (4.4.113)
ε1 = x − q1 =
ln(u)
ln(a)
− q1 =
ln(u) − q1 ln(a)
ln(a)
=
ln(u) − ln(aq1
)
ln(a)
=
ln u
aq1
ln(a)
=
ln y−1
y
q1
0
ln(a)
=
ln(y1)
ln(a)
(4.4.114)
1
ε1
=
ln(a)
ln(y1)
(4.4.115)
1
εn−1
= qn + εn ⇒ ε2 =
1
ε1
− q2 =
ln(a)
ln(y1)
− q2 =
ln(y0)
ln(y1)
− q2 (4.4.116)
log2(21
) log2(3) log2(22
) ⇒ log2(2) log2(3) 2 log2(2) ⇒ 1 log2(3) 2
(4.4.117)
ε1 =
ln(y−1)
ln(y0)
− q1 (4.4.118)
ε1 =
ln(3)
ln(2)
− 1 (4.4.119)
a
b
= log2(3) = q1 +
1
1/ε1
= 1 +
1
1
ln(3)
ln(2)
− 1
= . . . (4.4.120)
y0+1 = y1 =
y−1
yq1
0
=
3
21
=
3
2
(4.4.121)
ln(3)
ln(2)
− 1
−1
=
ln(2)
ln 3
2
=
ln(2)
ln(y1)
(4.4.122)

Draft
log2(3) = 1 +
1
ln(2)
ln(y1)
= . . . (4.4.123)
1
ln(2)
ln 3
2
2 (4.4.124)
ε2 =
ln(yn−2)
ln(yn−1)
− qn =
ln(y0)
ln(y1)
− q2
ln(2)
ln 3
2
− 1 =
ln(2) − ln 3
2
ln 3
2
=
ln 4
2
ln 3
2
= (4.4.125)
∼90 %
21 votes, 68.57%

Draft
4.5 Set Theory
D
Uring our study of numbers, operators, and number theory (in the chapters of the
same name), we often used the terms groups, rings, body, homomorphism,
etc. and thereafter we will continue to do it again many times. Besides the fact that
these concepts are of utmost importance, to give demonstrations or build mathemat-
ical concepts essential to the study of contemporary theoretical physics (quantum field physics,
string theory, standard model, ... ), they allow us to understand the components and the basic
properties of mathematics and its operators by storing them in separate categories. So, choose
to present the Set Theory as the 5th chapter of this book is a very questionable choice as rigor-
ously that it is where almost everything begins... However, we still needed to expose the Proof
Theory for the notations and methods that will be used here.
Moreover, when teaching modern mathematics in the secondary or primary (in the years 1970),
the language of sets and the preliminary study of binary relations to a more rigorous approach
to the notion of functions and applications of mathematics in general was introduced (see defi-
nition below).
Definition: We talk about arrow diagram ( or sagittal diagram from latin sagitta = arrow)
to all diagram showing a correspondence between the two sets of components connected wholly
or partially by a set of arrows.
For example, the graphical representation of a defined function of the set E =
{−3, −2, −1, 0, 1, 2, 3} to the set F = {0, 1, 2, 4, 9} lead to the sagittal diagram below:
Figure 4.28 – Sagittal Diagram example from a definition set to a destination set
A relationship from E to E provide an arrow diagram of the type:

Draft
Figure 4.29 – Function returning in its own set of definitions
The closure of each element showing a reflexive relation and the systematic presence of a
back arrow indicating a symetrical relation.
Definition: If the target set is identical to the original set, we say that we have a binary rela-
tion.
However choosing to introduce the Set Theory in school classrooms has also some other reason.
In fact, for the sake of internal rigour (i.e. not related to reality), a very large part of mathematics
was rebuilt within a single axiomatic framework, so called Set Theory, in the sense that each
mathematical concept (previously independent of the other) is returned to a definition where all
the logical components come from this same framework: it is regarded as fundamental! Thus,
the rigour of reasoning carried out within Set Theory is guaranteed by the fact that the frame is
non-contradictory or consistent. Let us see now the definitions that build this framework.
Definitions:
D1. We name set any list, collection or gathering of well-defined objects, explicitly or im-
plicitly.
D2. A Universe U is an object whose constituents are sets .
Note that what mathematicians name Universe is not a set! In fact it is a model
that satisfies to the axioms of sets.
Indeed, we will see that we can not talk about the set of all sets (because this is
not a set) to designate the object that consists of all the sets and that’s why we talk about
Universe.
D3. We name elements or members of the set objects belonging to the set and we write:
p ∈ A (4.5.1)
if p is an element of the set A and in the contrary case:
p ∈ A (4.5.2)
If B is a part of A, or subset of A, we write this:
B ⊂ A or A ⊃ B (4.5.3)

Draft
Thus:
∀x, x ∈ B ⇒ x ∈ A (4.5.4)
Examples:
E1. A = {1, 2, 3}
E2. X = {X | x is a positive integer}
D4. We can provide sets with a number of relations that compare (useful sometimes...) their
elements or to compare some of their properties. These relations are called comparison
relations or order relations (see section Operators).
Remarks
R1. The structure of ordered set has original been set up in the framework of the
Numbers Theory by Cantor and Dedekind .
R2. As we have proved in the chapter on Operators, N, Z, Q, R are totally ordered
by the usual relations ≤, ≥. The relation , often called strict order is not an
order relation because not reflexive and not antisymmetric (see section Operators). For
example, in N the relation a divides b , often denoted by the symbol | is a partial order.
R3. If R is an ordering on E and F is a subset of E, the restriction to F of the relation
R is an order on F, called order induced by R in F.
R4. If R is an order on E, the relation R defined by:
xR y ⇔ yRx
is an order on E, called reciprocal order of R. The reciprocal order ≤ of the usual order
is the order noted ≥ and reciprocal order to the order a divides b in N is the order b is
a multiple of a.
The set is the basic mathematical entity whose existence is defined: it is not defined as itself but
by its properties, given by the axioms. It uses a human process: a kind of categorization feature,
which allows thought to distinguish several independent qualified elements.
Theorem 4.20. We can demonstrate from these concepts, that the number of subsets of a set of
cardinal n is 2n
.
Proof 4.20.1. First there is the empty set ∅, that is 0 items Chosen from n, i.e. Cn
0 (notation of
binomial coefficient non-conform with ISO 31-11!) as we have seen in chapter Probabilities:
Ck
n = v =
n
k
=
Ak
n
k!
=
k!
k! (n − k)
(4.5.5)

Draft
and so on...
The number of subsets (cardinal) of E corresponds to the summation of all binomial coefﬁ-
cients:
Card (P (E)) =
n
k=0
Ck
n (4.5.6)
But, we have (see section Algebraic Calculation):
(x + y)n
=
n
k=0
Ck
nxk
yn−k
(4.5.7)
therefore:
Card (P (E)) =
n
k=0
Ck
n1k
1n−k
= (1 + 1)n
= 2n
(4.5.8)
Q.E.D.
Example:
Consider the set S = {x1, x2, x3}, we have the set of all parts of P(S) consisting
of:
− The empty set: {} = ∅
− The singletons: x1, x2, x3
− The duets: x1, x2, x1, x3, x2, x3
− Itself: {x1, x2, x3}
Such that:
P(S) = {∅, x1, x2, x3, x1, x2, x1, x3, x2, x3, x1, x2, x3} (4.5.9)
What makes effectively 8 elements!
Remark
The order in which the elements are differentiated does not come into account when
counting parts of the original set.

Draft
In Applied Mathematics, we work almost exclusively with sets of numbers. Therefore, we will
limit our study of definitions and properties of these.
Now let us formalize the basic concepts for working with the most common sets we encounter
in the basic school curriculum.
4.5.1 Zermelo-Fraenkel Axiomatic
The Zermelo-Fraenkel axiomatic, abbreviated sometimes ZF-C axioms shown below was
formulated by Ernst Zermelo and Abraham Adolf Fraenkel specified by the early 20th century
and completed by the axiom of choice (hence the capital C in ZF-C). It is considered as the
most natural axiomatic structure in the context of set theory.
Remark
There are many other axiomatic structures, based on the more general concept of class,
as developed by von Neumann, Bernays and Gödel (for the notations, see the chapter on
the Proof Theory).
Strictly technically speaking..., the ZF axioms are statements of calculation for first order pred-
icate (see section Proof Theory) egalitarian in a language with only one primitive symbol for
membership (binary relation). The following should therefore only be seen as an attempt (...)
to express in English the expected significance of these axioms.
A1. Axiom of extensionality:
Two sets are equal if and only if they have the same elements. This is what we note :
A = B ⇔ (∀x ∈ A, x ∈ B) ∧ (∀x ∈ B, x ∈ A) (4.5.10)
So A and B are equal if every element x of A is also in B and every element x of B also
belongs to A.
A2. Axiom of empty set:
The empty set exists, we note it:
∅ (4.5.11)
and it has no element, its cardinality is therefore 0. In fact this axiom can be deduced
from another axiom that we will see a little further but it is convenient to introduce it by
convenience for teaching in high-school classes.
A3. Axiom of pairing:
If A and B are two sets, then, there exist a set C containing A and B alone and as
components. This set C is then noted A, B. From the perspective of the sets considered
elements that gives:
∀A∀B∃C (A ∈ C ∧ B ∈ C) (4.5.12)

Draft
This axiom also shows the existence of the singleton a set noted:
{X} (4.5.13)
which is a set whose only element is X (and therefore with unitary cardinal). We simply
need to apply the axiom asking equality between A and B.
A4. Axiom of the sum (also called axiom of union):
This axiom allows us to build the union (merge) of sets. Said in a most common way: the
union (merge) of any family of a set, is... a set. The union of any family of sets is often
noted:
i
Ai (4.5.14)
or if we take some of its elements:
x∈A
xi (4.5.15)
A5. Axiom of subsets:
He expressed that for any set A, the set of all its parts P(A) exists (do not confuse with
the P of probability!). So for any set A, we can associate a set B which contains exactly
the parts C (verbatim the subsets) of the first:
∀A∃B∀C (C ∈ B ⇔ C ⊂ A) (4.5.16)
A6. Axiom of infinity:
This axiom express the fact that there exist an infinite set. To formalize it, we say that
there exist a set, called autosuccessor set A containing ∅ (the empty set) such that if x
belongs to A, then also x ∪ {x} belongs to A:
A is autosuccessor ⇔ (∅ ∈ A) ∧ (x ∈ A ⇒ (x ∪ {x}) ∈ A) (4.5.17)
This axiom expresses for example that the set of integers exists. Indeed, N is so the small-
est autosuccessor set in the sense of inclusion N = {∅, {∅, {∅, ...}}} and by convention
we note (where we build the Natural Set):
0 = ∅
1 = {∅}
2 = {∅, {∅}}
...
(4.5.18)
A7. Axiom of regularity (also called foundation axiom):
The main purpose of this axiom is just to eliminate the possibility of having A as part of
itself.
Thus, for any non-empty set A, there exists a set B which is an element of A such that no
element of A is an element of B (you must distinguish the level of the language used, a
set and its elements have not the same status!) that we note:
∀A = ∅ ⇒ ∃B ∈ A, A ∩ B = ∅ (4.5.19)
and thus result we expected to have:
∀A, A ∈ A (4.5.20)

Draft
Proof 4.20.2. Indeed, let A be a set such that A ∈ A. Consider the singleton {A}, set
whose only element is A. According to the axiom of foundation, we must have an element
of this singleton that has no element in common with him. But the only possible element
is A itself , that is to say that we must have:
A ∩ {A} = ∅ (4.5.21)
But by hypothesis A ∈ A and by construction A ∈ {A}. So:
A ∈ (A ∩ {A}) (4.5.22)
which contradicts the previous assertion. Therefore:
A ∈ A (4.5.23)
Q.E.D.
A8. Axiom of replacement (also called Axiom schema of replacement):
This axiom expresses the fact that if a formula f is a functional then for any set A, there
is a set B consisting precisely of the images of A by this function.
So, in a little more formally way, the set A of elements a and a binary relation f (which is
quite generally a functional), there exist a set B consisting of elements b such that f(a, b)
is true. If f is a function where b is not free then it means that:
b = f(A) and B = f(A) (4.5.24)
In a technical way we write this axiom as following:
∀A ∃a ∈ A ∃!f f(a, b) ⇒ ∃B∀a ∈ A∃b ∈ Bf(a, b) (4.5.25)
So for every set A and any item it contains, there is one and only one b defined by the
functional f such that there exists a set B for which any element a belonging to the set A
there is a b belonging to set B defined by the functional f.
Let’s see an example with the following binary predicate that for the value of any a from
A determines the value of any b of B:
P(a, b) = (a = 1 ∧ b = 2) ∨ (a = 3 ∧ b = 4) (4.5.26)
Therefore from the knowledge that a is equal 1 we derive that b is equal 2 and similarly
(i.e. by replacement) when a is equal 3, we derive that b is equal 4.
We see well through this small example the strong relationship that exists considerating
the predicate P as a naive function! Moreover, as there an infinity of possible functions
f, the replacement scheme is considered as an infinite number of axioms.
A9. Axiom of Selection (also called Axiom comprehension schema):
This axiom simply expresses that for any set A and any property P expressible in the
language of set theory, the set of all elements of A satisfying the property P exist.

Draft
So more formally, to any set A and any condition or proposition P(x), there is a set B
whose elements are exactly the elements x of A for which P(x) is true. This is what we
write:
B = {x ∈ A : P(x)} (4.5.27)
In a more comprehensive and rigorous way we have in fat for any functional f that does
not include a as free variable:
∀A∃B∀a (a ∈ B ⇔ a ∈ A ∧ f) (4.5.28)
It is typically the axiom that we use to construct the set of even numbers:
{a ∈ N | ∃b ∈ N, a = 2b} (4.5.29)
or to prove the existence of the empty set (which invalidates the axiom of the empty set)
because you just have to ask that there exist a set that satisfies the property:
A = A (4.5.30)
and regardless of the set A. And only the empty set satisfies this property by the selection
axiom.
The compliance with the strict conditions of this axiom eliminates the paradoxes of the
naive set theory, as Russell’s paradox or Cantor’s paradox who invalidated the naive set
theory.
For example, consider the Russell set R of all sets that do not contain themselves (note
that we give a property of R without specifying what is this set):
R = {E : E ∈ E} (4.5.31)
The problem is to know whether or not R contains itself or not. If R ∈ R, then, R is self-
contained, and, by definition R ∈ R, and vice versa. Each possibility is contradictory.
If we now denote by C the set of all sets (Cantor Universe) we have in particular:
P(C) ∈ C (4.5.32)
which is impossible (i.e. with the power of the continuum of real numbers), according to
Cantor’s theorem (see section Numbers).
These paradoxes (or syntactic antinomies) come from a non-compliance with the
conditions of application of the selection axiom: to define E (in the example of Russell),
there must be a proposition P which bears on the set R, which should be explicated. The
proposal defining the set of Russell or that of Cantor does not indicate what is the set E.
It is therefore invalid!
A very nice and well known example (this is why we present it) helps to better understand
(this is the Russel paradox which we have already spoken about int length in the section
on Proof Theory):
A young student went one day to his barber. He entered into conversation and asked him
if he had many competitors in his pretty city. Seemingly innocent way, the barber replied,
«I have no concurrence. Because of all the men of the city, I obviously do not shave

Draft
those who shave themselves, but I am fortunate to shave. all those who do not shave
themselves».
What then in such a so simple statement could take to the fault the logic of our young
smart student?
The answer is in fact innocent, until we decide to apply to the case of the barber: Does he
shaves himself, Yes or No?
Suppose he shaves himself: he then belongs to the category of those who shave them-
selves, those who the barber said he did of course not shave.... So he does not shave
himself..........
Finally, this unfortunate barber is in a strange position: if he shaves himself, he does
not shave himself, and if he does not shave himself, he shaves himself. This logic is
self-destructive, contradictory stupidly, rationally irrational.
Then comes the selection axiom: We exclude the barber of all persons to which the
declaration applies. Because in reality, the problem is that the barber is a member of the
set of all the men of the city. So what applies to all men does not apply to the individual
case of the barber.
A10. Axiom of Choice:
Given a set A of non-empty mutually disjoint sets, there exist a set B (the set of choices
for A) containing exactly one element for each member A.
However let us indicate that the issue of the axiomatization and therefore of the founda-
tions found himself still shaken by two questions at the time of their construction: what
valid axioms must be chosen and in a system of axioms are the mathematics coherent (do
we not have a risk of seeing a contradiction)?
The first issue was first raised by the continuum hypothesis: if we can put two sets of
numbers in correspondence term to term, they have the same number of elements (car-
dinal). We can thus map all integer numbers with rational numbers as we have shown
in the section on Numbers, so they have the same cardinality, be we can not map integer
numbers with all the real numbers. The question then is whether there is a set whose
number of elements would be located between the two or not? This question is important
to build the classical theory of analysis and mathematicians usually choose to say there is
none, but we can also say the opposite.
In fact the continuum hypothesis is linked in a more profound way we could thing to the
axiom of choice which can also be formulated as follows: if C is a collection of non-
empty sets then we can select any element of each set of the collection. If C has a finite
number of elements or a countable number of elements, the axiom seems pretty trivial:
we can sort and number the sets of C and the selection of an element in each set is simple.
Where it begin to get complicated is when the set C has the power of the continuum: how
to choose the elements if it is not possible to number them?
Finally in 1938 Kurt Gödel shows that set theory is consistent without the axiom of choice
and without the continuum hypothesis as well as with! And to end it all Paul Cohen in
1963 shows that the axiom of choice and the continuum hypothesis are not related.
Ok to make a pedagogical summary of all this stuff consider the following figure (excluding the
axiom of choice):

Draft
Figure 4.30 – Zermelo-Frankel axioms visual summary (source:?)
4.5.1.1 Cardinals
Definition: Sets are said to be equipotent if there exists a bijection (one-one correspondence)
between these sets. We thus say they have same cardinal that the norm ISO 3111 advocated
to write card(S) but in this book we will also use the notation Card(S) (many U.S. books use
non-official notation that looks exactly like the absolute value | S | or #S).
Thus, more rigorously, a cardinal (which quantifies the number of items in the set) is an equiv-
alence class (see section Operators) for the relation of equipotence.
Remark
Cantor is the main creator of set theory, in a form that we name today naive set theory.
But, apart to elementary considerations, his theory was also consisting of higher abstrac-
tion levels. The real novelty of the Cantor theory is that it lets talk about infinity. For
example, an important idea Cantor was precisely to define the equipotence.
If we write c1 = c2 as equality of cardinals, we mean by that there are two equipotent sets A

Draft
and B such that:
c1 = Card(A) and c2 = Card(B) (4.5.33)
Cardinals can be compared. The order thus defined is a total ordering (see section Operators)
between the Cardinals (the proof that the order relation is complete uses the axiom of choice
and the proof that it is antisymmetric is known under the name of Cantor-Bernstein’s theorem
that we will demonstrate later below).
Say that c1 c2 means in simple language that A is equipotent to a proper part of B, but B
is not equipotent to any own part of A. Mathematicians would say that Card(A) is smaller or
equal to the Card(B) if there is an injection of A into B.
We saw during our study of numbers (see section Numbers), especially of transfinite numbers,
that an equipotent set (or bijection) to N was told to countable set.
Let us now see this notion a little more in detail:
Let A be a set, if there is an integer n such that there is at least for each element of A a cor-
responding item int the set {1, 2, ..., n} (in fact this is rigorously a bijection... concept that we
will define later) then we say that the cardinal of A, denoted Card(A) or Card(A) is a finite
cardinal and its value is n.
Otherwise, we say that the set A has an infinite cardinal and we write:
Card(A) = +∞ (4.5.34)
A set A is countable if there is a bijection between A and (N). A set of numbers A is
countable if there is a bijection between A and part of (N). A set at maximum countable is
thus of finite cardinal, or countable.
We can therefore check the following proposals:
P1. A part of a countable set is at most countable.
P2. A set containing a non-countable set is also not countable.
P3. The product of two countable sets is countable.

Draft
Remark
We can restrict a set of numbers relatively with the null element and the negative or
positive elements in it and therefore we write (example for the real set):
R∗
= R {∅}
R+ = {x | x ∈ R and x ≥ 0}
R− = {x | x ∈ R and x ≤ 0}
R∗
+ = R+ {∅}
R∗
− = R− {∅}
(4.5.35)
These concepts are similar for N, Z, Q (the set of complex numbers C being not ordered,
the second and third line does not apply to).
So any infinite subset of N is equipotent to N itself, what may seem counter-intuitive at first...!
In particular, there are as many even integers as any natural integer numbers (use the bijection
f(n) = 2n) from N to P, where P is the set of even natural numbers. As many relative numbers
as integers, as many integers as rational numbers (see the section on Numbers for the proofs).
Thus we can write:
Card(N) = Card(Z) = Card(Q) = ℵ0 (4.5.36)
and more generally, any infinite part of Q is countable.
Thus we have an important result: any infinite set therefore has an infinite countable part.
Since we have proved in the section on Numbers that the set of real numbers has the power
of the continuum and that the set of natural numbers has transfinite cardinal ℵ0, Cantor raised
the question whether there was a cardinal between the transfinite cardinal ℵ0 and the cardinal of
R? In other words, we have an infinite amount of integers, and an even greater amount of real
numbers. So does it exist an infinite greater than the infinite of integers and smaller than that of
the real numbers?
The problem arose by writing ℵ0 the cardinal of N and ℵ1 (new) the cardinal of R and offering
to demonstrate or contradict that:
ℵ1 = 2ℵ0
(4.5.37)
according to the combinatorial law that gives the number of elements that we can get from from
all subsets of a set (as we have proved it before).
The rest of his life, Cantor tried, in vain, to prove this result that we name the continuum hy-
pothesis. He did not succeed and descended into madness. In 1900, the International Congress
of Mathematicians, Hilbert considered that this was one of the 23 major issues that should be
resolved in the 20th century.
This problem is solved in a rather surprising way. First, in 1938, one of the greatest logicians of
the 20th century, Kurt Gödel showed that the hypothesis of Cantor was not rebuttable, that is to

Draft
say, we could never prove that it was false. Then in 1963, the mathematician Paul Cohen closed
the debate. He demonstrated that we could never prove that it was true!!! We can conclude
rightly that Cantor had become mad to try to demonstrate a problem that could not be proved.
4.5.1.2 Cartesian Product
If E and F are two sets, we name Cartesian product of E by F the set noted E × F (not to
be confused with the vector product notation) consisting of all possible pairs (e, f) where e is
an element of E and f an element of F.
More formally:
E × F = {(e, f) | e ∈ E ∧ f ∈ F} (4.5.38)
We note the Cartesian product of E by itself:
E × E = E2
(4.5.39)
and then we say that E2
is the set of pairs of elements of E.
We can perform the Cartesian product of a sequence E1 × E2 × ... × En of sets and get all
n-tuples (e1, e2, ..., en) where e1 ∈ E1, e2 ∈ E2, ..., en ∈ En.
In the case where all sets Ei are identical to E, the Cartesian product is obviously noted En
.
We then say that en
is the set of all n-tuples of elements of E.
If E and F are finite then the Cartesian product E × F is finished. Moreover:
Card(E × F) = Card(E) · Card(F) (4.5.40)
From here we see that if the sets E1, E2, ..., En are finished then the Cartesian product E1 ×
E2 × ... × En is finished and we have:
Card(E1 × E2 × ... × En) =
n
i=1
Card(Ei) (4.5.41)
In particular:
Card(En
) = [Card(E)]n
(4.5.42)
if E is a finite set.
Examples:
E1. If R is the set of real numbers, then R2
is the set of all couples of real numbers. In the
plane reported to a referential, any point M has the coordinates that are an element of R2
.
E2. When we run two dice whose faces are numbered 1 through 6, each die can be
symbolized by by the set E = {1, 2, 3, 4, 5, 6}. The outcome of a roll of dies is then an
element of E2
= E×E. The Cardinal of E×E is then 36. There is therefore 36 possible
results when we launch two dices whose faces are numbered 1 to 6.

Draft
Remark
Set theory and the concept of cardinal is the theoretical basis of relational database soft-
wares.
4.5.1.3 Intervals
Let M be a set of any numbers so that M ⊂ R (particular but frequent example). We have for
definitions.
D1. x ∈ R is called upper bound of the set M, if x ≥ m for ∀m ∈ M. Conversely, we
speak about lower bound (so do not confuse the concept of terminal with the concept
of interval!).
D2. Either M ⊂ R, M = ∅. x ∈ R is called the smallest upper bound noted:
x = supM (4.5.43)
of M if x is an upper bound of M and if for any upper bound y ∈ R we have x ≤ y.
Conversely, we speak about the smaller lower bound that we note:
x = infM (4.5.44)
The definitions are equivalent in the context of functional analysis (see section of the same
name) as the functions are defined on sets.
Indeed, let f be a function whose domain of definition I swept all R. We note that:
f : E → R (4.5.45)
and let x0 ∈ R.
Definitions:
D1. We say that f has a global maximum on x0 if:
∀x ∈ E : f(x) ≤ f(x0) (4.5.46)
D2. We say that f has a global minimum on x0 if:
∀x ∈ E : f(x) ≥ f(x0) (4.5.47)
In each of these cases, we say that f has an global extremum on (it is a concept that we
often use in the sections of Analytical Mechanics and Numerical Methods!).
D3. f is upper bounded if there is a real number M such as ∀x ∈ I, f(x) ≤ M. In this case,
the function has an upper bound of f on its domain of definition I traditionally denoted:
sup
I
f (4.5.48)

Draft
D4. f is lower bounded if there is a real M such that ∀x ∈ I, f(x) ≥ M. In this case, the
function has a lower bound of f on its domain of deﬁnition I traditionally denoted:
inf
I
f (4.5.49)
D5. We say that f is bounded if it is both lower bounded and upper bounded (typically the
case of trigonometric functions).
4.5.2 Set Operations
We can build from at least three sets A, B, C all sets operations (which notations are due to
Dedekind) existing in set theory (very useful in the study of probability and statistics).
Remark
Some of the notations below will be frequently use later in relatively complex theorems,
so it is necessary to understand them deeply!
Thus, we can construct the following set operations:
4.5.2.1 Inclusion
In the simplest case, we deﬁne the inclusion as:
A ⊂ B ⇔ ∀x [x ∈ A ⇒ x ∈ B] (4.5.50)
In a non-specialized language here’s what to you have to read: A is included (is a part, or is
a subset) in B then for all x belonging to each of these x also belongs to B:
Figure 4.31 – Visual example (Euler Diagram) of the inclusion

Draft
where the U in the lower right corner of the figure represents the Cantor Universe.
From this it follows the following properties:
P1. If A ∈ B and B ∈ A then it implies A = B and vice versa.
P2. If A ∈ B and B ∈ C then implies A ∈ C.
4.5.2.2 Intersection
In the simplest case, we define the intersection as:
A ∩ B = {x | x ∈ A ∧ x ∈ B} (4.5.51)
In a non-specialized language here’s what you have to read: the intersection of sets A and B
consists of all the elements that are both in A and in B:
Figure 4.32 – Visual example (Euler Diagram) of the intersection
More generally, if (Ai) is a family of sets indexed by i ∈ I, the intersection of the (Ai), i ∈ I is
denoted:
i∈I
Ai (4.5.52)
This intersection is explicitly defined by:
i∈I
Ai = {x | ∀i ∈ Ix ∈ Ai} (4.5.53)
That is to say the intersection of the family of indexed sets includes all x that are located in each
set of all sets of the family.

Draft
Given two sets A and B, we say they are disjoint if and only if:
A ∩ B = ∅ (4.5.54)
Furthermore, if:
A ∩ B = ∅ ⇔ Card(A ∪ B) = Card(A) + Card(B) (4.5.55)
Mathematicians note that:
A B (4.5.56)
and name it disjoint union.
We sometimes joke that knowledge is built on the disjunction... (those who understand will
appreciate...).
Definition: An collection S = Si of non-empty sets form a partition of a set A if the following
properties hold:
P1. ∀Si, Sj ∈ S and i = j ⇒ Si ∩ Sj = ∅
P2. A =
Si∈S
Si
Example:
The set of even numbers and the set of odd numbers are a partitions of Z.
The intersection law is trivially a commutative law (see further below the definition of the
concept of law) as:
A ∩ B = B ∩ A (4.5.57)
4.5.2.3 Union
In the simplest case, we define the union (also sometimes named merge) as:
A ∪ B = {x | x ∈ A ∨ x ∈ B} (4.5.58)
In a non-specialized language here’s what you have to read: the union (or merge) of the sets
A and B is the set of elements that are in A plus those that are in B.

Draft
Figure 4.33 – Visual example (Euler Diagram) of the union
More generally, if (Ai) is a family of sets indexed by i ∈ I, the union of the (Ai), i ∈ I is
denoted:
i∈I
Ai (4.5.59)
This union is explicitly deﬁned by:
i∈I
Ai = {x | ∃i ∈ Ix ∈ Ai} (4.5.60)
That is to say that the union of the family of indexed sets includes all x for which there is a set
indexed by i such that x is included in on of the set Ai.
We have the following distributive properties:
i∈I
Ai ∩ B =
i∈I
(Ai ∩ B) (4.5.61)
i∈I
Ai ∪ B =
i∈I
(Ai ∪ B) (4.5.62)
The law of union ∪ is a commutative law (see further below the deﬁnition of the concept of
law) as:
A ∪ B = B ∪ A (4.5.63)
We also name idempotences laws the relations (note that for the general culture):
A ∩ A = A (4.5.64)
A ∪ A = A (4.5.65)
and absorptions laws the relations:
A ∩ (A ∪ B) = A (4.5.66)
A ∪ (A ∩ B) = A (4.5.67)

Draft
The laws of intersection and union are associative, such that:
A ∩ (B ∩ C) = (A ∩ B) ∩ C (4.5.68)
A ∪ (B ∪ C) = (A ∪ B) ∪ C (4.5.69)
and distributive such that:
A ∩ (B ∪ C) = (A ∩ B) ∪ (A ∩ C) (4.5.70)
A ∪ (B ∩ C) = (A ∪ B) ∩ (A ∪ C) (4.5.71)
If we recall the concept of cardinal (see above) we have with the previously deﬁned opera-
tions, the following relationship:
Card(A ∪ B) = Card(A) + Card(B) − Card(A ∩ B) (4.5.72)

Draft
4.5.2.4 Difference
In the simplest case, we deﬁne the difference as:
A B = {x | x ∈ A ∧ x ∈ B} (4.5.73)
In a non-specialized language here’s what you have to read: The difference of the sets A and
B consists of all the elements found only in A (and thus excluding those of B):
Figure 4.34 – Visual example (Euler Diagram) of the difference
4.5.2.5 Symmetric Difference
Let U be a set. For any equation we deﬁne the symmetric difference AδB between A and B
by:
A B = (A B) ∪ (B A) (4.5.74)
In a non-specialized language here’s what you have to read: The symmetric difference of the
sets A and B consists of all items that are only in A and those found only in B (we pass aside
elements that are common):

Draft
Figure 4.35 – Visual example (Euler Diagram) of the symetric difference
So as we can see we have:
A B = (A ∪ B) (A ∩ B) (4.5.75)
Some trivial properties are given below:
P1. Commutativity: A B = B A
P2. Complementarity (see definition below): Ac
Bc
= A B
4.5.2.6 Product
In the simplest case, we define the set product or cartesian product as:
A × B = {(x, y) | x ∈ A ∧ y ∈ B} (4.5.76)
In a non-specialized language here’s what to you have to read: product (not to be confused
with the multiplication or cross product of vectors) of two sets A and B is the set of pairs such
as each element of each set is combined with all elements of the other set.
The product set of real numbers for example generates the plane where each element is defined
by X and Y axis.
We often find products sets in mathematics and physics when we work with functions. For
example, a function of two real variables which gives real output will be written:
f(x, y)
R×R−→R
→ z (4.5.77)
or more simply:
f(x, y) = z (4.5.78)

Draft
4.5.2.7 Complementarity
In the simplest case, we deﬁne the complementarity as:
∀A ⊂ U ¯A = {x | x ∈ U, x ∈ A} (4.5.79)
In a non-specialized language here’s what you have to read: The complementary is deﬁned
as taking a set U and a subset A of U then the complement of A in U is the set of elements that
are in U but not in A:
Figure 4.36 – Visual example (Euler diagram) of the difference
Other notations of complementarity that is sometimes found in the literature and the following
book are (depending on the context to avoid confusion with other stuff):
U Ac
or Ac
(4.5.80)
or in the particular example above, we could also just write B A.
We have for properties for all Ai included in any B:
i∈I
Ai
c
=
i∈I
Ac
i (4.5.81)
i∈I
Ai
c
=
i∈I
Ac
i (4.5.82)
Here are some trivial properties regarding to complementarity:
¯¯A = A (4.5.83)
A ∩ ¯A = ∅ (4.5.84)
A ∪ ¯A = U (4.5.85)

Draft
There are other very important relations that also applied to Boolean logic (see section Logic
Systems). If we consider three sets A, B, C as shown below we have:
A (B ∩ C) = (A B) ∪ (A C) (4.5.86)
A B C = (A B) ∪ (A ∩ B) (4.5.87)
and the famous De Morgan’s laws in set form (see section Logic Systems), which are given
by the relations:
A ∩ B = ¯A ∪ ¯B
A ∪ B = ¯A ∩ ¯B
(4.5.88)
We would like indicate before moving on to another topic, that a signiﬁcant number of adults
in employment (mostly managers) having forgotten the previous deﬁned concepts after leaving
high school must study them again when they learn the SQL language (Structured Query Lan-
guage) which is the most common worldwide language to query corporate databases servers in
the 20th and 21st century. Most of them learn in training centers the following scheme to build
queries with joins:
Figure 4.37 – Common SQL query expressions with joins

Draft
4.5.3 Functions and Applications
Definition: In mathematics, an application (or function) denoted typically f - in analysis
- or A - in linear algebra - is the information of two sets, the departure set E and arrival set F
(or image of E), and a relation associating each element x of the departure set one and only
one element of the arrival set, which we call image of x by f in the analysis field we note that
f(x) or f(E) to explicit the departure set. We name images the elements of f(E) and the
elements of E are called the antecedents.
Then we say that f is an application from E to F denoted:
f : E → F (4.5.89)
(remember the first arrow/sagittal diagram presented at the beginning of this section), or we also
say that this is an application of arguments in E and values in F.
Remark
Note: The term function is often used for applications with scalar numeric values, real
or complex, that is to say when the arrival set is R or C. We speak then of real function
or complex function. In the case of vector we prefer to use the word application as
we already mention it in the definition.
Definitions:
D1. The graph (or also called graphic or representative) of an application or function
f : E → F is the subset of the cartesian product E ×F consisting of pairs (x, f(x)) for x
varying in E. The data of the graph f determines its starting set (by projection on the first
argument often denoted x) and image (projection on the second argument often denoted
y).
D2. If the triplet f(E, F, Γ) is a function where E and F are two sets and Γ ⊂ (E × F) is a
graph, E and F are the source and purpose of f respectively. The definition domain or
departure set of f is:
Df = I = {x ∈ E | ∃y ∈ F, (x, y) ∈ Γ} (4.5.90)
D3. Given three non empty sets E, F, G, any function of E ×F to G is named a composition
law of E × F with values in G.
D4. An internal composition law (or simply internal law) in E is a composition law of
E × E with values in E (that is to say this is the case E = F = G).
Remark
The subtraction in N is not an internal composition law although it is part of the four
basic high-school arithmetic operators. But the addition in N is such an internal
law.

Draft
D5. An external composition law (or simply external law) in E is a composition law of
F × E with values in E, where F is a separate set of E. In general, F is a set, called
scalar set.
Example:
In the case of a vector space (see definition much lower) the multiplication
of a vector (whose components are based on a given set) by a real scalar is an
example of external composition law.
Remark
An external composition law with values in E is also called action of F on E.
The set F is then the field operators. They also say that F operates on E (keep in
mind the example of the vectors mentioned above).
D5. We name image of f, and note Im(f), the subset defined by:
Im(f) = f(E) = {y ∈ E | ∃x ∈ E, y = f(x)} (4.5.91)
Thus, the image of a fonction f : E → F is the collection of f(x) for x browsing E. It
is a subset of F.
And we name kerr of f, and we note ker(f), the very important subset in mathematics
defined by:
ker(f) = f({0}) = {x ∈ E | f(x) = 0} (4.5.92)
According to the figure (you must deeply understand this concept because we will reuse
the ker many times to prove theorems that have important practical applications later in
various chapters):
Figure 4.38 – ker concept of a function

Draft
Remark
R1. ker(f) is derived from the German Kern, simply meaning kernel.
R2. Normally the notations Im and ker are reserved for group homomorphisms,
rings, fields and to linear applications between vector spaces and modules, etc. (see
further below). We do not usually use them for any applications between any sets.
But ... it does not really matter for the moment at this level of the book.
Applications and functions can have a phenomenal amount of properties. Below you can found
some easy one that are part of the general knowledge of the physicist (for more information
about what a function is, see the section on Functional Analysis).
Let f be an application or function of a set E to a set F then we have the following properties:
P1. An application or function is said to be surjective if:
Any element y of F is the image by f of at least (we emphasize on the at least)
an element of E. We thus say that it is a surjection from E to F. It follows from
this definition, that an application or function f : E → F is surjective if and only if
F = Imf. In other words, we also write this definition as following:
∀y ∈ F, ∃x ∈ E : y = f(x) (4.5.93)
Figure 4.39 – Schematic representation of a surjective application or function
P2. An application or function is said to be injective if:
Any element y of F is the image by f of at most f (we emphasize the at most)
a single element of E. We thus say that f is an injection of E to F. It follows from
this definition, that an application or function f : E → F is injective if and only if
the relations x1, x2 ∈ E and f(x1) = f(x2) involve. In other words: an application or
function for which two separate elements have distinct images is called injective. Or an
application or function is injective at least if one of the following equivalent properties
holds:

Draft
P2.1 ∀x, y ∈ E2
: f(x) = f(y) ⇒ x = y
P2.2 ∀x, y : x = y ⇒ f(x) = f(y)
P2.3 ∀y ∈ F the equation in x, y = f(x) has at least one solution in E
All this can be resumed by:
Figure 4.40 – Schematic representation of an injective application or function
P3. An application or function is said to be bijective or total application/function if:
An application or function f from E to F is both injective and surjective. In this
case, we have that for any element y of F, the equation y = f(x) admits in E a single
(not at least or not at most) pre-image x. What we also write:
∀y ∈ F, ∃!x ∈ E : y = f(x) (4.5.94)
This is illustrated by:
Figure 4.41 – Schematic representation of a bijective application or function
We are thus naturally led to deﬁne a new application from F to E, called inverse func-
tion or reciprocal function of f and noted f−1
that to every element of F matches
the unique pre-image element x of E (also called sometimes solution) of the equation
y = f(x). In other words:
x = f−1
(y) (4.5.95)

Draft
The existence of an inverse (reciprocal) function or application implies that the graph of
a bijective function or application (in the set of real numbers...) and that of its inverse
(reciprocal) are symmetric with respect to the right of equation y = x.
Indeed, we notice that if y = f(x) is equivalent to x = f−1
(y), then these equations
imply that the point (x, y) is on the graph of f if and only if the point (y, x) is the graph
of equation f−1
.
As you can see for example in the ﬁgure below with the sinus function (see section
Trigonometry):
Figure 4.42 – Bijective function example

Draft
Example:
Take the case of a holiday station where a group of tourists must be housed
in a hotel. Each way to allocate these tourists in hotel rooms may be represented
by an application of all tourists to all the rooms (to each tourist is assigned a
room).
• Tourists want the application to be injective, that is to say, each of them has a
single room. This is only possible if the number of tourists does not exceed
the number of rooms.
• The hotel manager hopes that the application is surjective, that is to say, each
room is occupied. This is only possible if there are at least as many tourists
than rooms.
• If it is possible to spread the tourists so that there is only one per room, and
all the rooms are occupied: the application will then be both injective and
surjective that is to say bijective.
Remarks
R1. It comes from the deﬁnitions above that f is bijective in the set of real numbers
if and only if any horizontal line intersects the graph of the function at a single
point. This leads us to the second following remark:
R2. An application that satisﬁes the test of the horizontal line is continuously
increasing or decreasing at any point in its domain.
P4. An application or function is called composite application or composite function if:
Let ϕ be an application or function from E to F and ψ an application or function of F in G.
The application or function that associates to each element x of the set E an element ψ(ϕ(x))
of G is named composed application of ϕ and ψ and is denoted by:
ψ ◦ ϕ (4.5.96)
where the symbol ◦ is called round (do not confused with the scalar product we will see
later in the section of Vectorial Calculus). Thus, the above relation is written psi round phi
but has to be read phi round psi (...). So:
(ψ ◦ ϕ)(x) = ψ(ϕ(x)) (4.5.97)
Let, moreover, χ be an application (not a function!) of G in H. We check immediately that the
composition operation is associative for applications (for more details see the section of Linear
Algebra):
χ ◦ (ψ ◦ ϕ) = (χ ◦ ψ) ◦ ϕ (4.5.98)

Draft
This allows us to omit parentheses and write more simply:
χ ◦ ψ ◦ ϕ (4.5.99)
In the particular case where ϕ would be an application or function from E to E, we note ϕk
the
composed application ϕ ◦ ϕ ◦ ... ◦ ϕ (k times).
What’s important in what we have seen until now in this section is that all deﬁned properties
listed above are applicable to Numbers’ Sets.
Let us see a concrete and very powerful example:

Draft
4.5.3.1 Cantor-Bernstein Theorem
Warning! This theorem, for which the result may seem trivial, is not necessarily easy to ap-
proach (its mathematical formalism is not very aesthetic...). We advise you to read the proof
slowly and imagine the sagittal diagrams in your head during the development.
Here is the hypothesis to prove:
Theorem 4.21. Let X and Y be two sets. If there is an injection (remember the definition of
an injective function or application above) from X to Y and another from Y to X, then both
sets are in bijection (remember the definition of an bijective function or application above). It
is therefore an antisymmetric relation.
This is illustrated by:
Figure 4.43 – Representation of a antisymmetric relation
For the proof we need rigorously to demonstrate beforehand a lemma (intuitively obvious... but
not formally) who’s statement is as follows:
Lemma 4.21.1. Let X, Y, Z three sets such that X ⊆ Z ⊆ Y . If X and Y are in bijection
through a function f, then X and Z are in bijection through a function g.
An example of application of this lemma is the set of natural numbers and rational numbers
which are in bijection (see the section of Number Theory for the proof). Therefore, all the
rational numbers are in bijection with the set of natural numbers since N ⊆ Z ⊆ Q.
Proof 4.21.1. First, formally, we create a function f from Y to X such that it is bijective:
f : Y → X (4.5.100)
To continue we need a set A that will be defined by the union of the images of the functions of
the functions f (of the kind f(f(f...)))) of the pre-images of the set Z (remember that Z ⊆ Y )
which we exclude elements of X (that we will be noted for this proof: Z − X).
In other words (if the first form is not clear...) we define the set A as the union of images of
(Z − X) by the applications f ◦ f ◦ ... ◦ f. What we write:
A =
∞
n=1
fn
(Z − X) (4.5.101)

Draft
Because f : Y → X and that (Z − X) ⊆ Y we have by construction A ⊆ X and thus
((Z − X) ∪ A) ⊆ Z. Note that we also have:
A =
∞
n=1
fn
(Z − X) ⇒ f(A) = f
∞
n=1
fn
(Z − X) =
∞
n=1
f (fn
(Z − X)) =
∞
n=1
fn+1
(Z − X)
(4.5.102)
and by reindexing:
f(A) =
∞
n=2
fn
(Z − X) (4.5.103)
We then have (make a pattern in your head of the arrow diagrams can help at that level of the
proof...):
f((Z − X) ∪ A) = A (4.5.104)
We can elegantly demonstrate this last relationship:
f((Z − X) ∪ A) = f(Z − X) ∪ f(A) = f(Z − X) ∪
∞
n=2
fn
(Z − X) =
∞
n=1
fn
(Z − X)
(4.5.105)
Since Z can be partitioned (nothing stop us to do this!) in two disjoint subsets (Z −X)∪A and
(X − A) and without forgetting that X ⊆ Z ⊆ Y and A ⊆ X, we set as deﬁnition the function
g (we don’t give more information about it yet) such that:
g : Z → X (4.5.106)
and for every pre-image a of g of the partition ((Z − X) ∪ A) ⊆ Z we have:
∀a ∈ ((Z − X) ∪ A) → f(a) (4.5.107)
This means that because ((Z−X)∪A) ⊆ Z and Z ⊆ Y we can thus apply the bijective function
f (remember that f : Y → X) as equivalent of the function g to any element of ((Z − X) ∪ A).
We also have also for every pre-image a of g of the partition (X − A) (remember that A ⊆ X):
∀a ∈ (X − A) → a (4.5.108)
The application g is then bijective because its restrictions to the ((Z − X) ∪ A) and (X − A)
are f and the identity which are bijective by deﬁnition.
Finally there exists, by construction, a bijection between X and Z.
Q.E.D.
Now that we have proved the Lemma let us recall the assumptions of the Cantor-Bernstein
theorem using the result of the Lemma:

Draft
Consider ϕ an injection from X to Y and ψ an injection for Y to X with X ⊆ Y .
We thus have:
ϕ(X) ⊆ Y and ψ(Y ) ⊆ X (4.5.109)
so (we recognize here the statement of the lemma):
ψ(ϕ(X)) ⊆ ψ(Y ) ⊆ X (4.5.110)
a ◦ (b ∗ c) = (a ◦ b) ∗ (a ◦ c)
(a ∗ b) ◦ c = (a ◦ b) ∗ (b ◦ c)
(M, ∗) est un magma si =⇒
∗ est une opération
∗ est une loi interne
x ∗ a = x ∗ b =⇒ a = b
( ¨M, ∗) est un monoïde si =⇒
∗ est associative
∃ un élément neutre n ∈ ¨M pour ∗
a + b = c
a
i=1
1 +
b
i=1
1 =
a+b
i=1
1
a
i=1
1 +
b
i=1
1 + c =
a+b
i=1
1 + c
= 0

Draft
a+b
i=1
1 = −c
a + b = −c
a · a = n = 1
a =
1
a
(G, ∗) est un groupe ⇐⇒



∗ est associative
∃ ∈ G un élément neutre pour ∗
∀x ∈ G possède un symétrique
pour ∗
Q =
p
q
(p, q) ∈ Z/q = 0
(p/q)/r = p/(q/r)
a
b
=
b
a
= ±1
=⇒ a = ±b
(a + ib) + (c + id)
= a + ib + c + id
= a + c + ib + id
= (a + c) + i(b + d)

Draft
(a + ib) − (c + id)
= a + ib − c − id
= a − c + ib − id
= (a − c) + i(b − d)
(a + ib)(c + id)
= ac + aid + ibc + ibid
= ac + iad + ibc + i2
bd
= ac + iad + ibc + (−1)bd
= (ac − bd) + i(ad + bc)
1
x + iy
=
(x − iy)
(x + iy)(x − iy)
=
x − iy
x2 − (iy)2
=
x − iy
x2 + y2
=
x
x2 + y2
− i
y
x2 + y2
(A, +, ∗) est un anneau si ⇐⇒



(A, +) est un groupe abélien
La loi ∗ est associative
La loi ∗ est distributive par
rapport à la loi +
(Z, +)
(Q, +)
(R, +)
(C, +)
(Z, +, ×)
(Q, +, ×)
(R, +, ×)
(C, +, ×)

Draft
(C, +, ∗) est un corps si =⇒
(C, +, ∗) est un anneau unitaire
(C − {0}, ∗) est un groupe
(Z, +, ×)
(Q, +, ×)
(R, +, ×)
(C, +, ×)
(Q, +, ×)
(R, +, ×)
(C, +, ×)



(R, +) groupe commutatif



L’addition est une opération interne
L’addition est associative
L’addition est commutative
Il existe un élément neutre pour l’addition : 0
Tout nombre réel possède un opposé
(R, ×) groupe commutatif



La multiplication est une opération interne
La multiplication est associative
La multiplication est commutative
Il existe un élément neutre pour l’addition : 1
Tout nombre réel non nul possède un inverse
La multiplication est distributive par rapport à l’addition
(R, ≤) ensemble totalement ordonné



La relation ≤ est réﬂexive
La relation ≤ est antisymétrique
La relation ≤ est transitive
(E, +, ∗)



(E, +) est un groupe abélien
∗ est une loi externe déﬁnie par
∗ : E × K → E
(x, a) → a ∗ x

Draft
x = t2
+ 2t + 3
y = −t + 5
F = ∅
∀(X, Y ) ∈ F2
, X + Y ∈ F
∀λ ∈ K, ∀X ∈ F, λX ∈ F
F = ∅
∀(−→x , −→y ) ∈ F2
, −→x + −→y ∈ F
∀λ ∈ K, ∀−→x ∈ F, λ−→x ∈ F
(A, C, +, ·, ×) est une C-algèbre =⇒



(A, +, ·)est un C-espace vectoriel
(A, +, ×)est un anneau unitaire
∀λ ∈ C, ∀a, b ∈ A,
(λa) · b = a · (λb) = λ(a · b)
f(a ∗ b) = f(a) ◦ f(b) ∀a, b ∈ A
f(a ∗ b) = f(a) ◦ f(b) ∀a, b ∈ A
f(1A) = 1B
f(1A) = 1B
f(a + a ) = f(a) + f(a )
f(a · a ) = f(a)f(a )
ker(f) = {0}

Draft
f(x + y) = f(x) ∗ f(y) ∀x, y ∈ A
f(1A) = 1B
f(x−1
) = (f(x))−1
∀x ∈ A
∀x ∈ A : 1B = f(1A)
= f(x + x−1
)
= f(x) ∗ f(x−1
)
∀x ∈ A : 1B = f(x) ∗ f(x−1
)
∀x ∈ A : f(x) =
1B
f(x−1
= (f(x−1
))−1
1 = f(1) = f(a · b) = f(a) · f(b)
f(x + y) = f(x) + f(y)∀(x, y) ∈ A
f(λx) = λf(x) ∀x ∈ A, ∀λ ∈ K
f(a + a ) = f(a) + f(a )
= 0 + 0 = 0
f(ra) = f(r)f(a) = f(r) · 0 = 0
{ax|x ∈ A}

Draft
a = qr + r
I = rZ
m · a = d · n · a ∈ dZ
f(n) = f(1 + ... + 1)
n
= 1A + ... + 1A)
n
= n · 1A
f(n) = n · 1A
(1, 0) · (0, 1) = 0
∼95 %
16 votes, 62.5%

Draft
4.6 Probabilities
P
Robability is the measure of the likelihood that an event will occur and therefore the
calculation of probabilities handles random phenomena (known more aesthetically
as stochastic processes when their are time-dependent), that is to say, phenomena
that do not always lead to the same outcome and that can be studied using numbers
their implications and occurrences. However, even if these phenomena have variable outcomes,
depending on chance, we observe a certain statistical regularity.
Probability is quantified as a number between 0 and 1 (where 0 indicates impossibility and
1indicates certainty). The higher the probability of an event, the more certain we are that the
event will occur.
The concepts related to probabilities have been given an axiomatic mathematical formaliza-
tion in probability theory (see further below), which is used widely in such areas of study
as mathematics, statistics, finance, gambling, science (in particular physics), artificial intelli-
gence/machine learning, computer science, game theory, and philosophy to, for example, draw
inferences about the expected frequency of events. Probability theory is also used to describe
the underlying mechanics and regularities of complex systems.
Definitions: There are several ways to define a probability. Mainly we are talking about:
D1. Experimental or inductive probability which is the probability derived from the whole
population.
D2. Theoretical or deductive probability which is the known probability through the study of
the underlying phenomenon without experimentation. It is therefore an a priori knowl-
edge as opposed to the previous definition that was rather referring to a notion of a
posteriori probability.
As it is not always possible to determine a priori probabilities, we are often asked to perform
experiments. We must then be able to pass from the first to the second solution. This passage is
supposed to be possible in terms of limit (with a population sample whose size approaches the
size of the whole population).
The formal modeling of the probability calculus was invented by A.N: Kolmogorov in a book
published in 1933. This model is based on the probability space (U, A, P) that we will define a
little further and that we can relate to the theory of measurement (see section Measure Theory).
However, the probabilities were studied in the scientific point of view by Fermat and Pascal in
the mid 17th century.
Remark
If you have a teacher or trainer who dare to teach statistics and probabilities with exam-
ples based on gambling (cards, dice, match, toss, etc.) dispose it to whom it may concern
because it would mean that he has no experience in the field and he will teach you any-
thing and no matter how (examples could normally be based on industry, economy or
RD, in short: areas daily used in companies but especially not on gambling ...!).

Draft
4.6.1 Event Universe
Definitions:
D1. The universe of events, or universe of observables, U is the set of all possible out-
comes (results), called elementary events that occur during a random determined test.
The universe can be finite (countable) if the elementary events are finite or continuous
(uncountable) if they are infinite.
D2. Any event A is a set of elementary events and is part of the universe of possible U. It is
possible that an event is composed of only a single elementary event.
Example:
Consider the universe of all possible blood groups, then the event A the individ-
ual is Rh positive is represented by:
A = {A+, B+, AB+, O+} ⊂ U
while the event B the individual is the universal donor is represented by:
B = {O−} ⊂ U
thus being an elementary event.
D3. Let U be a universe and A an event, we say that the event A occurs (or is realised) if
during the run of the trial the issue i (i ∈ U) occurs and that i ∈ A . Otherwise, we say
that A was not realised.
D4. The empty subset ∅ of U is called impossible event. Indeed, if during a trial where the
event i occurs, we always have i ∈ ∅ and the event ∅ then never occurred.
If U is finite, or countably infinite, any subset of U is an event, that is no longer
true if U is uncountable (we will see in the chapter Statistics why).
D5. The set U is also called certain event. Indeed, if at the end of the trial the event i occurs,
we have always (since U is the universe of events). The event U then always occurred.
D6. Let A and B be two subsets of U. We know that the events A ∪ B and A ∩ B are both
subsets of U then, events that are respectively joint events and disjoint events.
If two events A and B are such that:
A ∩ B = ∅ (4.6.1)

Draft
the two events may not be feasible during the same trial, then we say that they are mutually
exclusive events.
Otherwise, if:
A ∩ B = ∅ (4.6.2)
the two events may be feasible during the same trial (the possibility to see a black cat when we
pass under a ladder, for example), we say conversely that they are independent events.
4.6.2 Kolmogorov’s Axioms
The probability of an event is somehow responding to the notion of frequency of a random
phenomena, in other words, at each event we will attach a real number in the interval [0, 1],
which measure the probability (chance) of realization. The properties of frequencies we can
highlight during various trials allow us to determine the properties of probabilities.
Let U be a universe. We say that we deﬁne a probability on the events of U if any event A of U
we associate a number or measure P(A), called a priori probability of event A or marginal
probability of A.
A1. Fon any event A:
1 ≥ P(A) ≥ 0 (4.6.3)
Thus, the probability of any event is a real number between 0 and 1 inclusive (this is common
human sense...).
A2. The probability of the certain event or of the set (sum) of possible events is equal to 1:
P(U) = 1 (4.6.4)
A3. If A ∩ B = ∅ two events are incompatible (disjoint), then:
P(A ∪ B) = P(A) + P(B) (4.6.5)
the probability of the merge (or) of two mutually incompatibles events (or mutually exclusive)
is therefore equal to the sum of their probabilities (law of addition). We then speak of disjointed
probability.
We understand better that the third axiom requires A ∩ B = ∅ otherwise the sum of all proba-
bilities could be greater than 1 (imagine again the set diagram of the two events in your head!).

Draft
Example:
Consider that in a given area, over 50 years, the probability of a major earthquake
is 5% and on the same period the probability a major flood is 10%. We would like to
know what is the probability that a nuclear plant meets at most one of two events during
the same period if they are incompatible.... We have then the total probability that is the
sum of the two probabilities which is 15%...
We will find an example of this kind of disjoint probability in the chapter of Industrial Engi-
neering when studying F.M.E.A. (Failure Modes and Effects Analysis) for fault analysis systems
with a complex structure.
In other words in a more general form if (Ai)i∈N is a sequence of pairwise disjoint events (Ai
and Aj can not occur at the same time though i = j) then:
P
n
i=1
Ai =
n
i=1
P(Ai) (4.6.6)
We then speak of σ-additivity because if we look more closely at the three axioms above the
measure P forms a σ-algebra (see section Measure Theory).
At the opposite, if the events are not incompatibles (they can overlap or in other words: they
have a joint probability), we then have for probability that at most one of the two takes place:
P(A ∪ B) = P(A) + P(B) − P(A ∩ B) (4.6.7)
This means that the probability that at most one of the events A or B occurs is equal to the sum
of the probabilities for the realization of A or B occurred, minus the probability that A and B
occurred simultaneously (we will show later that this is simply equal to the probability that the
two do not occur at the same time!).
Example:
is 5% and on the same period the probability a major flood is 10%... We would like to
know what is the probability that a nuclear plant meets at most one of two events during
the same period if the are not incompatibles. We then calculate the probability that from
the above equation that gives 14.5%...
And thus if they were incompatibles we would have then A ∩ B = ∅ we find again the disjoint
probability:
P(A ∪ B) = P(A) + P(B) (4.6.8)

Draft
An immediate consequence of the axioms (A2) and (A3) is the relations between the probability
of an event A and its complement, noted ¯A (or more rarely in accordance with the notation used
in the chapter of Proof Theory the complementary may be noted ¬A):
P( ¯A) = 1 − P(A) (4.6.9)
Let U be a universe with a finite number of n possible outcomes:
U = {i1, i2, ..., in} (4.6.10)
where the events:
I1 = {i1} , I2 = {i2} , I3 = {i3} , ..., In = {in} (4.6.11)
are called elementary events. When these events have the same probability, we say they are
equiprobables. In this case, it is very easy to calculate the probability. Indeed, these events
being by definition incompatible with each other at this level of our discussion, we have under
the third axiom (A3) of probabilities:
P (I1 ∪ I2 ∪ ... ∪ In) = P(I1) + P(I2) + ... + P(In) (4.6.12)
but since:
P (I1 ∪ I2 ∪ ... ∪ In) = P(U) = 1 (4.6.13)
and that the probability of the right hand are by hypothesis equiprobable, we have:
P(I1) = P(I2) = ... = P(In) =
1
n
(4.6.14)
Definition: If A and B are not mutually exclusive but independent, we know that by their
compatibility A ∩ B = ∅, that (very important in statistics!):
P(A ∩ B) = P(A) · P(B) (4.6.15)
the probability of the intersection (and operator) of two independent events is equal to the
product of their probabilities (law of multiplication). We name it joint probability (this is the
most common case).
Example:
is 5% and on the same period the probability a major flood is 10%... Assume that these
two events are not mutually exclusive. In other words that they are compatible. We will
be interested to their independence. Thus, we would like to know what is the probability
that a nuclear power plants meets the two events at the same time, at any time, during
this same period. We then calculate the probability from the above equation that gives
0.05%...

Draft
Under a more general form, the events A1, A2, ..., An are independent if the probability of the
intersection is the product of the probabilities:
P
n
i=1
Ai =
n
i=1
P(Ai) (4.6.16)
Remark
Be careful to not confuse independent and incompatible!
So far to summarize a bit we have:
Type Expression
2 incompatibles events (disjoints) P(A ∪ B) = P(A) + P(B)
2 not incompatibles events (joints) P(A ∪ B) = P(A) + P(B) − P(A ∩ B)
2 not incompatibles but independents events P(A ∩ B) = P(A) · P(B)
Table 4.14 – Classical cases of probabilities
Thanks the above definition, we can show that the probability that either A or B is to take place
(e.g. at least one of the two but not both at the same time), is simply equal to... the probability
that the two do not does not occur at the same time:
P(A ∪ B) = P(A) + P(B) − P (A ∩ B))
= P(A) + P(B) − P(A)P(B) = 1 − P( ¯A)P( ¯B)
= 1 − P(1 − A)P(1 − B)
(4.6.17)
We can also use this definition to determine the probability that only one of two events occurs:
P(A ⊕ B) = P(A)P( ¯B) + P(B)P( ¯A)
= P(A)(1 − P(B)) + P(B)(1 − P(A)) = P(A) + P(B) − 2P(A)P(B)
= P(A) + P(B) − 2P(A ∩ B)
(4.6.18)
Example:
is 5% and on the same period the probability a major flood is 10%.... We would like to
know what is the probability that a nuclear power plant exactly meets one of the both
events during the same period, assuming they can not occur at the same time. We then
calculate the probability from the above equation and that gives 14%...
There is a common and important area in the industry where the four following relations are
frequently used:

Draft
AND = P(A) · P(B)
OR COMPATIBLE = P(A) + P(B) − P(A ∩ B)
OR INCOMPATIBLE = P(A) + P(B)
XOR = P(A) + P(B) − 2P(A ∩ B)
(4.6.19)
This is the tree analysis error or probabilistic tree analysis which is used to analyse the
possible reasons for failure of a system of any kind (industrial, administrative or other).
To close this part of the chapter consider the following figure displaying Venn diagrams (see
section Set Theory) for all 16 events (including the impossible event) that can be described in
terms of two given events A and B. In each case, the event is represented by the red area:
Figure 4.44 – Possible Venn diagrams for two events
Consider the situation where A represents and earthquake and B represents a major flood and U
the universe of all dramatics events for a nuclear power plant. We consider that the two events
are independents. Then each of the 16 events can be described as follows, either mathematically
or verbally.
1. An earthquake can occur or a flood or nothing or the both together or any other event (to
resume: any event can occur).
P(U) = 1 = 100% (4.6.20)

Draft
2. A ∪ B: Any event with an earthquake a flood or the both event together can occur.
P(A ∪ B) = P(A) + P(B) − P(A ∩ B) = P(A) + P(B) − P(A)P(B) (4.6.21)
3. A ∪ Bc
: Any event with earthquake can occur with or without a flood excepted events
with a flood not associated to an earthquake.
P(A ∪ Bc
) = P(U) − P(B) + P(A ∩ B) = 1 − P(B) + P(A)P(B) (4.6.22)
4. Ac
∪ B: Any event with earthquake can occur with or without a flood excepted events
with a flood not associated to an earthquake.
P(Ac
∪ B) = P(U) − P(A) + P(A ∩ B) = 1 − P(A) + P(A)P(B) (4.6.23)
5. Ac
∪Bc
: Any event can occur excepted those associated with an earthquake together with
a flood.
P(Ac
∪ Bc
) = P(U) − P(A ∩ B) = 1 − P(A)P(B) (4.6.24)
6. A: Any event with an earthquake can occur (this include the events associating an earth-
quake and a flood).
P(A) = P(A) (4.6.25)
7. B: Any event with a flood can occur (this include the events associating a flood and an
earthquake).
P(B) = P(B) (4.6.26)
8. (A∩B)∪(Ac
∩Bc
): Any event can occur excepted those including an earthquake without
a flood or those including a flood without an earthquake.
P((A ∩ B) ∪ (Ac
∩ Bc
)) = P(U) − P(A) − P(B) + 2P(A ∩ B)
= 1 − P(A) − P(B) + 2P(A)P(B)
(4.6.27)
9. (A∩Bc
)∪(Ac
∪B): Any event including an earthquake without a flood or a flood without
an earthquake can occur.
P((A ∩ Bc
) ∪ (Ac
∪ B)) = P(A) + P(B) + 2P(A ∩ B) (4.6.28)
10. Bc
: Any event excepted those including a flood can occur.
P(Bc
) = P(U) − P(B) = 1 − P(B) (4.6.29)
11. Ac
: Any event excepted those including an earthquake can occur.
P(Ac
) = P(U) − P(A) = 1 − P(A) (4.6.30)
12. A ∩ B: Any events associating an earthquake and a flood together can occur.
P((A ∩ B)) = P(A)P(B) (4.6.31)

Draft
13. A ∩ Bc
: Any event with an earthquake and without a flood can occur.
P((A ∩ Bc
)) = P(A) − P(A)P(B) (4.6.32)
14. Ac
∩ B: Any event with a flood and without an earthquake can occur.
P(Ac
∩ B) = P(B) − P(A)P(B) (4.6.33)
15. Ac
∩ Bc
: Any event can occur excepted those including an earthquake and/or a flood.
P(Ac
∩ Bc
) = P(U) − P(A ∪ B) = 1 − P(A) − P(B) − P(A)P(B) (4.6.34)
16. A ∩ Ac
or B ∩ Bc
: Impossible Event.
P(A ∩ Ac
) = P(B ∩ Bc
) = P(∅) = 0 (4.6.35)
4.6.3 Conditional Probabilities
What can we infer about the probability of an event B knowing that an event A has occurred,
aware that there is a link between A and B? In other words, if there is a link between A and B,
the completion of A has to change our understanding of B and we want to know if it is possible
to define the conditional probability of an event (relatively) to another event.
This type of probability is called conditional probability or a posterior probability of B
knowing A, and is denoted in the context of the study of conditional probabilities:
P(B/A) (4.6.36)
and often in practice to avoid confusion with a possible division:
P(B | A) (4.6.37)
and we sometimes find in U.S. books the notation:
P(B ∧ A) (4.6.38)
or also:
PB(A) (4.6.39)
We also have the case:
P(A/B) (4.6.40)
which is called likelihood function of A or a priori probability of A given B.
Historically, the first mathematician to have used the correct notion of conditional probability
was Thomas Bayes (1702-1761). This is why we often say Bayes or Bayesian probabilities
as soon as conditional probabilities are involved: Bayes formula, Bayesian statistics,etc.

Draft
The notion of conditional probability that we will introduce is much less simple than it first
appears and the conditionals problems are an inexhaustible source of errors of any kind (there
are famous paradoxes on the subject and even expert requires peer review to minimize mistakes).
Let’s start with a simple example: Suppose we have two dice. Now imagine that we only
launched the first die. We want to know what is the probability that by throwing the second
dice, the sum of the two numbers is equal to a given minimum value. Thus, the probability
of obtaining the minimum value given the value of the first die is totally different from the
probability of obtaining the same minimum value in throwing two dice at the same time. How
to calculate this new probability?
Let us now formalize the process! After the launch of the first dice, we have:
A = {the result of the first throw is...} (4.6.41)
Under the hypothesis that B ⊂ A , we feel that P(B/A) must be proportional to P(B), the
proportionality constant being determined by the normalization:
P(A/A) = 1 (4.6.42)
Now let B ⊂ Ac
(B is included in the complement of A so that the events are mutually exclu-
sive). It is then relatively intuitive .... that under the previous hypothesis of incompatibility we
have the conditional probability:
P(B/A) = 0 (4.6.43)
This leads us to the following definitions of respectively a posteriori and a priori probabilities:
P(B/A) =
P(A ∩ B)
P(A) and P(A/B) =
P(A ∩ B)
P(B) (4.6.44)
Thus, the fact to know that A has occurred reduces all possible outcomes of the universe U of
B. From there, only the events of type A ∩ B are important. The probability of A given B or
vice versa (by symmetry) must be proportional to P(A ∩ B)!
The coefficient of proportionality is the denominator and it ensures the certain event. Indeed, if
two events A and B are independent (think the black cat and the scale for example), then we
have:
P(A ∩ B) = P(A)P(B) (4.6.45)
and then we see that P(B/A) is equal to P(B) and therefore the event A adds no information
to B and vice versa! So in other words, if A and B are independent, we have:
P(B/A) = P(B) and P(A/B) = P(A) (4.6.46)
Another fairly intuitive way to see things is to represent the probability measure P as a measure
of subsets areas (surface) of R2
.
Indeed, if A and B are two subsets of respective areas P(A) and P(B) then the question of
what is the probability that a point in the plane belongs to B knowing that it belongs to A it is
quite obvious that the probability is given by answer:
Surface(A) ∩ Surface(B)
Surface(A)
(4.6.47)

Draft
We would like to indicate that the deﬁnition of conditional probability is often used in the
following way:
P(A ∩ B) = P(A/B)P(B) = P(B/A)P(A) (4.6.48)
call formula of compound probabilities. Thus, the a posteriori probability of B knowing A
can also be written as:
P(B/A) =
P(A/B)P(B)
P(A) (4.6.49)
Example:
Suppose a disease like meningitis. The probability of having the meningitis will
be denoted by P(M) = 0.001 (arbitrary value for the example) and a sign of this
disease like headache will be noted P(S) = 0.1. We assume known that the a posteriori
probability of having a headache if we have meningitis is:
P(S/M) = 0.9 (4.6.50)
The Bayes’ theorem then gives the a priori probability of having meningitis if we have a
headache!:
P(M/S) =
P(S/M)P(M)
P(S)
= 0.09 (4.6.51)
We also note that:
P(A) = P ((B1 ∪ B2 ∪ ... ∪ Bi) ∩ A)
= P ((B1 ∩ A) ∪ (B2 ∩ A) ∪ ... ∪ (Bi ∩ A))
= P ((B1 ∩ A) + (B2 ∩ A) + ... + (Bi ∩ A))
= P(A/B1)P(B1) + P(A/B2)P(B2) + ... + P(A/Bn)P(Bn) = sumn
i=1P(A/Bi)P(Bi)
(4.6.52)
So we can know the probability of the event A knowing the elementary probabilities P(Bi) of
its causes and the conditional probabilities of A for each Bi:
P(A) =
n
i=1
P(A/Bi)P(Bi) (4.6.53)
which is called the formula of total probabilities or total probabilities theorem. But also,
for any j, we have the following corollary using the previous results that gives us following an
event A, the probability that it is the cause Bi that produced it:
P(Bj/A) =
P(Bj ∩ A)
P(A)
=
P(A/Bj)P(Bj)
i
P(A/Bi)P(Bi) (4.6.54)

Draft
which is the general form of the Bayes formula or Bayes’ theorem that we will us a little
in the Statistical Mechanics chapter and through the study of the theory of queues (see section
Quantitative Management). You should know that the implications of this theorem are, however,
considerable in daily life, in medicine, in industry and in the field of Data Mining.
We often find in the literature many examples of applications of the previous relationship with
only two possible outcomes B with respect to the event A. Therefore we find the Bayes formula
written in the following form for each issue:
P(B1/A) =
P(A/B1)P(B1)
P(A/B1)P(B1) + P(A/B2)P(B2)
=
P(A/B1)P(B1)
P(A/B1)P(B1) + P(A/ ¯B1)P( ¯B1)
P(B2/A) =
P(A/B2)P(B2)
P(A/B1)P(B1) + P(A/B2)P(B2)
=
P(A/B2)P(B2)
P(A/ ¯B2)P( ¯B2) + P(A/B2)P(B2)
(4.6.55)
and note that in this particular case (binary outcomes):
P(B1/A) + P(B2/A) = P(B1/A) + P( ¯B1/A)
=
P(A/B1)P(B1)
P(A/B1)P(B1) + P(A/ ¯B)P( ¯B)
+
P(A/B2)P(B2)
P(A/ ¯B2)P( ¯B2) + P(A/B2)P(B2)
=
P(A/B1)P(B1)
P(A/B1)P(B1) + P(A/ ¯B)P( ¯B)
+
P(A/B2)P(B2)
P(A/B1)P(B1) + P(A/ ¯B1)P( ¯B1)
=
P(A/B1)P(B1) + P(A/B2)P(B2)
P(A/B1)P(B1) + P(A/ ¯B1)P( ¯B1)
=
P(A/B1)P(B1) + P(A/ ¯B1)P( ¯B1)
P(A/B1)P(B1) + P(A/ ¯B1)P( ¯B1)
= 1
(4.6.56)
is an intuitive result.
For binary events, we also have (returning to the theorem of total probabilities seen above):
P(A) =
n
i=1
P(A/Bi)P(Bi)
= P(A/B1)P(B1) + P(A/B2)P(B2)
= P(A/B1)P(B1) + P(A/ ¯B1)P( ¯B1)
(4.6.57)

Draft
Examples:
E1. A disease affects 10 people on 10’000 (0.1% = 0.001). A test has been de-
veloped which has a 5% false positives (people not having the disease but for which the
test says they are affected) but still always detects the disease if a person has it. What is
the probability that a random person for which the test gives a positive result really has
this disease?
There is therefore 10,000 people, 500 of which are false positives, and we know a
posteriori that 10 people have really the disease. Then the probability that somebody
who has a positive test result is really sick is:
P(M) = 0.001
P( ¯M) = 0.999
P(T/M) = 1
P(T/ ¯M) = 0.05
P(T) = P(T/M)P(M) + P(T/¯(M))P(¯(M) = 0.05095
P(M/T) =
P(T/M)P(M)
P(T)
=
1˙0.001
0.05095
= 0.19627
(4.6.58)
This is often a shocking and counter-intuitive result. It also highlights why diagnostic
tests must be extremely reliable!
E2. Two machines M1 and M2 produce respectively 100 and 200 pieces. M1
produces 5% defective pieces and M2 produces 6% (posterior probabilities). What is
the a priori probability that a defective piece was manufactured by the machine M1? We
then have:
P(Mj/A) =
P(A/Mj)P(Mj)
i
P(A/Mi)P(Mi)
=
P(A/M1)P(M1)
P(A/M1)P(M1) + P(A/M2)P(M2)
=
5
100
300
100
5
100
100
300
+
6
100
200
300
=
5%
1
3
5%
1
3
+ 6%
2
3
∼= 39%
(4.6.59)
E3. From a batch of 10 pieces with 30% defective, we take a sample of size 3 without
replacement. What is the probability that the second piece is correct (whatever the ﬁrst
is)?
We have:
P(A) =
n
i=1
P(A/Bi)P(Bi) =
2
i=1
P(A/Bi)P(Bi)
= P(A/B1)P(B1) + P(A/B2)P(B2) =
7
9
3
10
+
6
9
7
10
= 70%
(4.6.60)

Draft
E4. We conclude with an important example for companies where employees have
more time in their career to pass exams or assessments in the form of multiple choice
questions (MCQs). If an employee responds to a question there are two issues: either he
knows the answer or he try to guess it. Let p be the probability that the employee know
the answer and therefore 1 − p that he guess it. We admit that the employee who guesses
will correctly answer with a probability of 1/m where m is the number of proposed
answers. What is the a prori probability that an employee (really) knows the answer to a
question with 5 choices if he answered correctly?
Let B and A be respectively the events the employee knows the answer and the em-
ployee correctly answers the question. Then the a priori probability that an employee
knows (really) the answer to a question that he answered correctly is:
P(B/A) =
P(A/B)P(B)
P(A/B)P(B) + P(A/ ¯B)P( ¯B)
=
1 · p
1 · p +
1
m
(1 − p)
=
5
6
∼= 83%
(4.6.61)
Bayesian analysis provides also a powerful tool to formalize reasoning under uncertainty and
the examples we have shown above illustrate how this tool can be difficult to use.
4.6.3.1 Conditional Expectation
Now, we will see to the continuous version of the conditional probability by introducing the
subject directly with a particular example (the general theory being indigestible) infinitely im-
portant in the field of social statistics and quantitative finance. However, this choice (the study
of a particular case) implies that the reader has read the first chapter of Statistics to study the
functions of continuous distributions and especially that of the Pareto law.
So here’s the scenario: Often, in social sciences or economics, we find in the literature dealing
with the laws of Pareto statements like the following (but almost never with a detailed proof):
whatever your income, the average income of those who have an income above yours is in a
constant ratio, greater than 1, to your income if it follows a Pareto random variable. Then we
say that the law is isomorphic to any truncated part itself.
Let X be a random variable equal to the income and following a Pareto with the density (see
section Statistics):
Let’s see what it is exactly:
f(x) = k
xk
m
xk+1
(4.6.62)
with k 1, xm 0, x ≥ xm and that has for distribution function (see also the Statistics
chapter for the detailed proof):
P(X ≤ x) = 1 −
xm
x
k
(4.6.63)

Draft
The sentence begins with whatever your income ..., then select any income x0 ≥ xm.
Now we need to compute the average income of those with income higher than x0. It is
therefore asked to calculate the expected (average income) of a new random variable Y that is
equal to X, but restricted to the population of people with an income above x0:
Y = X|{X≥x0} (4.6.64)
The distribution function of Y is given by:
P(Y ≤ x) = P (X ≤ x | X ≥ x0) (4.6.65)
This expression is of course equal to zero if x ≤ x0. Well, so far we have only do vocabulary.
First recall the following conditional probability relation already seen before:
P(B/A) =
P(A ∩ B)
P(A)
(4.6.66)
for x ≥ x0 we have for the conditional law:
P (X ≤ x | X ≥ x0) =
P(x0 ≤ X ≤ x)
P(X ≥ x0) (4.6.67)
Before going further, you should be aware that the numerator and denominator are independent
but that the whole must be considered, however, as the realization of a single random variable
which we denote Y . Furthermore, only the numerator is a dependent variable. The denominator
can it be considered as a normalization constant.
So we see that the density of Y is given by the function:
fY (y) =



0 y x0
f(y)
P(X ≥ x0)
y ≥ x0
Now we can calculate the expectation of Y :
E(Y ) =
ˆ
yfY (y)dy =
+∞ˆ
x0
f(y)
P(X ≥ x0)
dy =
1
P(X ≥ x0)
+∞ˆ
x0
yf(y)dy
=
1
P(X ≥ x0)
+∞ˆ
x0
yk
xk
m
yk+1
dy =
kxk
m
P(X ≥ x0)
+∞ˆ
x0
1
yk
dy
=
kxk
m
P(X ≥ x0)
1
(k − 1)yk−1
+∞
x0
=
1
P(X ≥ x0)
kxk
m
(k − 1)xk−1
0
(4.6.68)
Knowing that (see section Statistics):
P(X ≥ x0) =
+∞ˆ
x0
k
xk
m
xk+1
dx =
xk
m
xk
0
=
xm
x0
k
(4.6.69)

Draft
We finally have:
E(Y ) =
k
k − 1
x0 (4.6.70)
E(Y ) represents also the average income of those with an income above x0 and as can be seen
from the above equality it is in a constant ratio, greater than 1, to your income x0.
We can check this result by doing a Monte Carlo simulation in a spreadsheet software (it is
interesting to mention it to generalize to situations not computable by hand). You just need to
simulate the inverse of the distribution function:
X =
xk
m
1 − P(X ≤ x)
1/k
(4.6.71)
in MS Excel 11.8346:
a@6f6U¢6f6T/(1-RANDBETWEEN(1;10000)/10000))ˆ(1/$B$6)
and then take the average of the values obtained above or equal to a given X (which corresponds
to x0) and ensure that we get the good results as proved above!
Obviously, we could also calculate the conditional variance (verbatim the conditional standard
deviation). It may come one day...
4.6.3.2 Bayesian Networks
Bayesian networks are simply a graphical representation of a problem of conditional probabil-
ities to better visualize the interaction between the different variables when they begin to be in
large numbers.
This is a technique increasingly used in decision aided software (Data Mining), artificial intel-
ligence (A.I.) and also in the analysis and risk management (ISO 31010 norm).
Bayesian networks are by definition directed acyclic graphs (see section Graphs Theory), so that
an event can not (even indirectly) influence its own probability, with quantitative description of
dependencies between events.
These graphs are used for both knowledge representation models and calculating conditional
probabilities machines. They are mainly used for diagnosis (medical and industrial), risk anal-
ysis (diagnostics failures, faults or accidents), spam detection (Bayesian filter), voice text and
image opinions analysis, fraud detection or bad payers as well as data mining (M.K.M.: Mining
and Knowledge Management) in general.

Draft
Remark
Many systems and software based on drawings or on information in existing databases
exists to build and analyse Bayesian networks. Paid solutions: SQL Server, Oracle,
Hugin. Free solutions (at this date): Tanagra, Microsoft Belief Network MSBNX 1.4.2,
RapidMiner. Personally I prefer the simplicity of the small software MSBNX from Mi-
crosoft. For information, in 15 years of professional experience as a consultant I have
met so far only one company on more than 800 multinationals which used Bayesian
networks... (in transportation).
Use a Bayesian network is assimilated to do Bayesian inference. Based on observed informa-
tion, we calculate the probability of possible known data but not observed.
For a given domain (e.g. medical), we describe the causal relationships between variables of
interest by a graph (we do not need again to specify that it is acyclic). In this graph, the causal
relationships between variables are not deterministic, but probabilistic. Thus, the observation
of a cause or multiple causes does not always implies the effect or effects that depend on it, but
only changes the probability of observing them.
The particular interest of Bayesian networks is to consider simultaneously a priori knowledge
of experts (in the graph) and experience contained in the data.
Example of 5 variables with relations (directed acyclic graph) and numbering of states/variables:
Figure 4.45 – Example of Bayesian network (acyclic oriented) with 5 states

Draft
Obviously, the construction of the causal graph is based primarily on return of experience (REX)
and sometimes results on standards or reports of expert committees. In computing science, the
causal graph automatically change depending on the content of databases (think at the Amazon
book store in real time target advertisements based on your past purchases or at the Genius
Apple service). But we can rarely think to all possibilities and there will sometimes hidden
states between two known states that have been forgotten and that would have allowed to better
modelize the situations.
Suppose in the example above that with the help of a corporate database, we know that in about
100, 000 man-days, we hat in the company 1, 000 accidents (i.e.. 1% of total) and 100 machines
failures (i.e.. 0.01%of total). Then we represent it in the traditional form as follows:
Figure 4.46 – Directed acyclic Bayesian network with probability of departure
where we have the subset S2, S4, S5 which is what experts name a serial or linear connection,
the triplet S3, S2, S4 is a called a divergent relationship (if the arrows were reversed for the
triplet, we would have a converging relationship).
Before going further with our example we will make some observations in relation to these
three types of relationships:
For clarity, we distinguish ﬁrst conditional independence and conditional dependence.
We say that events A and C are conditionally independent if given an event B the following

Draft
equality holds:
P(A/B) = P(A/B, C) (4.6.72)
So the term conditional implies the presence of B and the fact that C does not inﬂuence the
probability of the event A.
About conditional dependence, this time we can distinguish three types of relationships.
1. The conditional dependence of the following type is called a serial or linear connection
(already mentioned above):
Figure 4.47 – Serial/linear Bayesian network
where A, B and C are dependent (in this particular example there are 3 dependent nodes
A, B and C, but in general this dependence relates to all nodes if there were more than 3)
In addition, A and C are conditionally dependent to B. But if the variable B is known, A
no longer provides any useful information about C (the path of uncertainty is somehow
broken) and therefore A and C become conditionally independent. We then have the
conditional probability that simplify as follows:
P(C/B, A) = P(C/B) (4.6.73)
2. The conditional dependence of the following type is called a divergent connection (as
already mentioned above):
Figure 4.48 – Divergent Bayesian network

Draft
In addition, B and C are conditionally dependent on A. But if A is known, B does not
provide any more information on C (again the path of uncertainty is somehow broken)
and therefore B and C become independent. We then have for example if A is known:
P(C/A, B) = P(C/A) (4.6.74)
3. The conditional dependence of the following type is called a convergence connection
or V-Structure (as already mentioned above):
Figure 4.49 – Convergent Bayesian network
where this time the parents are independent. So B and C are independent but become
conditionally dependent on A. If A is known, then we have:
P(A/B, C) = P(A/B) (4.6.75)
The dependence between parents therefore requires the observation of their common
child.
Now to make a concrete example, suppose our database gives us (thanks to quality managers
who always inputs the quality issues) that when a machine failure occurred, 99 times out of 100
(99%) there has been a total production stop (i.e. 1% of time there was no production stop) and
on all stop production 1% was not due to a machine failure. What we traditionally represent as
follows:

Draft
Figure 4.50 – 1st level Bayesian network
So the implicit probability that there is a production stop is given by:
P(S4) = P(S4/S2)P(S2) + P(S4/ ¯S2)P( ¯S2) = 99% · 0.1% + 1% · 99.9% ∼= 1.098%
(4.6.76)
This value represents the implicit proportion of productions stop from the 100,000 man-days
(so we can give a proportion of rows in the database that represents a production stop regardless
of the cause and even without knowing the details of the database)..
It then follows immediately that the implicit probability that there is no production stop is given
by:
P(S4) = 1 − P( ¯S4) ∼= 98.902% (4.6.77)
This is consistent with what gives us the freeware MSBNX 1.4.2:

Draft
Figure 4.51 – Beginning of the Bayesian network with MSBNX 1.4.2
Now suppose we observed a production stop. What is the a posteriori probability that it is due
to a machine failure? We then have:
P(S2/S4) =
P(S4/S2)P(S2)
P(S4)
=
99% · 0.1%
1.098%
∼= 9.02% (4.6.78)
We can also check this with the software MSBNX 1.4.2:
Figure 4.52 – A Posteriori probability of a production stop due to a machine failure in MSBNX 1.4.2
Now, imagine that our database gives us (always thanks to quality managers who ensured to

Draft
input quality issues) that 99 times out of 100 (99%) when there was a production stop, there
was an evacuation. However 5% of evacuations were identiﬁed as having nothing to do with a
production stop (i.e. 95% of evacuations are due to ﬁre exercises OR other events):
Figure 4.53 – 2nd level Bayesian network
Now to calculate the implicit probability retrospectively (a posteriori) of evacuations compared
to machines failures, we saw that when we had a conditional dependence serie, the conditional
probability depends only on the direct parent. Thus, we get:
P(S5) = P(S5/S4)P(S4) + P(S5/ ¯S4)P( ¯S4) = 99% · 1.089% + 5% · 98.092% ∼= 6.03%
(4.6.79)

Draft
Figure 4.54 – Implicit probability of an evacuation in MSBNX 1.4.2
So the implicit probability of evacuation does not actually depend on machine failures.
Now suppose we have observed an evacuation. We want to know what is the a posteriori prob-
ability that it is due to a machine failure! Then we have:
P(S4/S5) =
P(S5/S4)P(S4)
P(S5)
=
99% · 1.098%
6.03%
∼= 18.02% (4.6.80)
Figure 4.55 – A Posteriori probability of an evacuation due to a machine failure in MSBNX 1.4.2
Now we study the case with the alarm and again a database allows us to build a table with
different probabilities:

Draft
Figure 4.56 – 2nd level Bayesian network with second branch
Now to calculate the implicit probability that there is an alarm, we will have to consider the four
possible situations. We then use the theorem of total probability:
P(S3) = P(S3/S1, S2)P(S1)P(S2) + P(S3/ ¯S1, S2)P( ¯S1)P(S2)
+ P(S3/S1, ¯S2)P(S1)P( ¯S2) + P(S3/ ¯S1, ¯S2)P( ¯S1)P( ¯S2)
(4.6.81)
What a little more rigorously should be written:
P(S3) = P(S3/S1 ∧ S2)P(S1)P(S2) + P(S3/ ¯S1 ∧ S2)P( ¯S1)P(S2)
+ P(S3/S1 ∧ ¯S2)P(S1)P( ¯S2) + P(S3/ ¯S1 ∧ ¯S2)P( ¯S1)P( ¯S2)
(4.6.82)
The numerical application therefore provides for the implied probability of an alarm:
P(S3) = 75% · 1% · 0.1% + 99% · 99% · 0.1% + 10% · 1% · 99.9%
+ 10% · 99% · 99.9% ∼= 10.089%
(4.6.83)
What can be built and check as follows with MSBNX 1.4.2:

Draft
Figure 4.57 – Implicit probability in MSBNX 1.4.2
It may be useful to the reader to know that he can sometimes found in the literature the following
notation:
P(S3/S1 = Yes, S2 = Yes) = P(S3/S1, S2) = P(S3/S1 ∧ S2) (4.6.84)
Remark
In the particular example studied above all event have only two states. But in practice
they can have 3, 4 and more states. Therefore probabilities cross-tables quickly become
enormous.
As in the previous case, suppose we know that there was a working accident. We wish then
calculate the a priori probability of an alarm. We then have (observe that the probability depends
actually only to the state S2 state since the state S1 is completely known!):
P(S3/S1) = P(S3/S1, S2)P(S2) + P(S3/S1, ¯S2)P( ¯S2)
= 75% · 0.1% + 10% · 99.9% = 10.065%
(4.6.85)

Draft
Figure 4.58 – Implicit probability of an alarm in MSBNX 1.4.2
So, knowing that there was a work accident increases the probability that there is an alarm (we
start from a probability of 10.089% to go to a probability of 10.65%).
To complete this example, we would calculate the a posteriori probabilities P(S2/S3) and
P(S1/S3). To do this, we must first calculate the a priori probabilities P(S3/S2) and
P(S3/S1) (this last one has been calculated just before).
We have for the missing value (which can be easily checked as before with MSNBX 1.4.2
software):
P(S3/S2) = P(S3/S2, S1)P(S1) + P(S3/S2, ¯S1)P( ¯S1)
= 75% · 1% + 99% · 99% ∼= 98.76%
(4.6.86)
We then have:
P(S3/S1) = 10.065%
P(S3/S2) = 98.76%
(4.6.87)
We now have everything we need to calculate the a priori probability of P(S2/S3) and
P(S1/S3):
P(S2/S3) =
P(S3/S2)P(S2)
P(S3)
=
98.76% · 0.1%
10.089%
= 0.9789%
P(S1/S3) =
P(S3/S1)P(S1)
P(S3)
=
10.065% · 1%
10.089%
= 0.9976%
(4.6.88)
So the a priori probability that there is a machine breakdown when we know that there is an
alarm is 0.979% (i.e. 0.021% that the trigger of the alarm is not a priori due to a machine
failure). Respectively there is, a priori, 0.998% probability that there is a work accident when
we know there is an alarm (and then 0.002% that it is not a priori due to a work accident).
From the critical point of view, when there is finally an alarm we can not say a lot of things....
This is because, in this case, to the fact that the events of significant interest both have low
probability to occur (work accident and machine failure) and that the employees respond quite
well at the start of the alarm (otherwise if the a priori probabilities were high it would mean
that the behavior of the employees is not good because we can guess - with exasperation - in
advance which problem occurs with a good confidence).

Draft
Remark
We did not find how to check these last calculations with MSNBX 1.4.2. If someone
finds how to do, it would be great to give us the details of the process.
To conclude, the reader will have noticed that the calculations can quickly become annoying
when the graph becomes complex and this explains the use of computer software. Furthermore,
in the banking sector that uses for example Bayesian networks for credit risk, the a priori prob-
ability can be more complex. For example we might want to know the a priori probability that
there is a machine failure knowing that we have an alarm and an accident:
P(S2/S3, S1) = P(S2/S3 = Yes, S1 = Yes) (4.6.89)
4.6.4 Martingales
A martingale in probabilities (there is another one in stochastic processes) is a technique to
increase the chances of winning in gambling while respecting the rules of the game. The princi-
ple is completely dependent on the type of game we are focusing, but this term is accompanied
by an aura of mystery that some players would know efficient secret techniques to cheat with
chance. For example, many players (or candidates to play) search THE martingale that will beat
the bank in the most common games in casinos (institutions whose profitability relies almost
entirely on the difference - however small - between the chances of winning and losing).
Many martingales are the dream of their author, some are actually inapplicable, some could
actually give the possibility to cheat a little. Gambling in general are unfair: whatever the shot
played, the probability of winning of the casino (or of the State in the case of a lottery) is
greater than this of the player. In this type of game, it is not possible to inverse the chances, just
to minimize the probability of gambler’s ruin.
The most common example is the roulette wheels martingale. It consists to play a single chance
to the roulette wheels (red or black, odd or even) to win, for example, a unit in a series of moves
by doubling his bet if we lose, and that until we earn. Example: the player bets 1 unit on red,
if red comes out, it stops playing and won 1 unit (2 units less gain setting unit), if black comes
out, he doubles his betting by 2 units on red and so on until he wins.

Draft
Figure 4.59 – Casino roulette wheel
Having a chance on two to win, he may think he will eventually win, and when he wins, he is
necessarily paid for everything he has played more one unit of his initial bet.
This martingale appears to be safe in practice. Note that in theory, to be sure of winning, we
should have the opportunity to play an unlimited number of times.... This has major drawbacks:
This martingale is in fact limited by the bets that the player can do because you have to double
the bet every time you lose: 2 times the initial bet, then 4, 8, 16 .... if he loses 10 times, he must
be able to bet 1024 times its initial investment for the 11th party! Therefore a lot of money for
little gain!
The roulette wheels also have a 0 which is neither red nor black. The risk of losing at every
shot is is then larger than 1/2...
In addition, to paralyze this strategy, casinos offer table games per set: from 1 to 100.-, from 2
to 200.-, from 5 to 500.-, ... Therefore it is impossible to use this method on a large number of
shots, which increases the risk of losing it all.
Blackjack is a game that has winning strategies: several playing techniques, which usually
require to memorize the cards, can overturn the chances in favour of the player. The mathe-
matician Edward Thorp has published in 1962 a book that was at the time a real best-seller. But
all these methods require long training weeks and are easily detectable by the croupier (sudden
changes in the amounts of bets are typical). The casino has then the opportunity to banish from
its establishment the players using this playing martingale.
It should be noted that there are enough advanced methods. One of them is based on the less

Draft
played combinations. In games where the gain depends on the number of winning players
(Lotto...), playing the least played combinations maximize gains. This is how some people sell
combinations that would be statistically very rarely used by other players.
Based on this reasoning, we can still conclude that a player who would have been able to
determine statistically the least played combinations, to maximize its expected payoff, will
in fact certainly not be the only player to have achieved this by the analysis of these famous
combinations! This means that in theory the numbers the least played are actually overplayed
combinations, the best might be to achieve a mix of played numbers and overplayed numbers
to play for the ideal combinations Another conclusion to all this is maybe that the best is to
play random combinations which ultimately are less likely to be chosen by the players who
incorporate a human and harmonious factor in the choice of their numbers.
4.6.5 Combinatorial Analysis
Combinatorial analysis (counting techniques) is the field of mathematics that deals with the
study of all the issues, events or facts (distinguishable or indistinguishable) with their arrange-
ments (combinations) ordered or not according to some given constraints.
Definitions:
D1. A sequence of objects (events, issues, objects, ...) is said ordered if each suite with a
particular order of objects is recognized as a particular configuration.
D2. A sequel is unordered if and only if we are interested in the frequency of appearance of
objects regardless of their order.
D3. The objects (of a sequence) are said distincts if their characteristics can not to be con-
fused with the other objects.
Remark
We chose to put combinatorial analysis in this chapter because when we calculate prob-
abilities, we also often need to know what is the probability of finding a combination or
arrangement of given events under certain constraints.
Students often have difficulty remembering the difference between a permutation, an arrange-
ment and a combination. Here is a little summary of what we’ll see:
• Permutation: We take all the objects.
• Arrangement: We choose objects from the original set and the order intervenes.
• Combination: Same as for the arrangement, but the order does not interfere.
You must not forget that for each result, the reverse will give the probability of falling respec-
tively on a given permutation/arrangement/combination!

Draft
We will present and demonstrate below the 6 most common cases from which we can find
(usually) all others:
4.6.5.1 Simple Arrangements with Repetition
Definition: A simple arrangement with repetition is an ordered sequence of length m of n
distinct objects not necessarily all different in the sequence (either: with possible repetitions!).
Let A and B be two finite sets of respective cardinal m, n such that there is trivially m ways to
choose an object in A (of type a) and n ways to choose an object in B (of type b).
We saw in the section Set Theory that if A and B are disjoint, that:
Card(A ∪ B) = Card(A) + Card(B) = m + n (4.6.90)
We therefore deduce the following properties:
P1. If an object can not be at the same time of type a and type b and if there is m ways to
select an object of type a and n ways to choose an object of type b, then the union of
objects gives m + n selections (this is typically the result of the SQL UNION queries
without filters in corporate Relational Databases Management System).
P2. If we can choose an object type of type a in m ways then an object of type b in n ways,
then there is according to the Cartesian product of two sets (see section Set Theory)
Card(A × B) = Card(A) · Card(B) = m · n (4.6.91)
ways to choose a single object of type a then an object of type b.
With the same notation for m and n, we can choose for each element of A, its single image
among the n elements of B. So there are n ways to choose the image of the first element of A,
then also n ways to choose the image of the second element of A, ..., and n ways to choose the
image of the m-th element of A . The total number of consecutive possible applications from A
to B is thus equal to the m product of n (thus m times the cartesian product of the cardinality
of the set B with itself!). It is usual to write it under the following way (we have indicated the
different ways to write his result as it can be found in various textbooks):
Card(BA
) = Card

B × B × ... × B
m times

 = Card(B)m
= ¯Am
n = nm
(4.6.92)
where BA
is the set of applications from A to B. The increase in the number of possibilities is
geometric (not exponential as it is often wrongly said!).
This result is mathematically similar to the ordered result (an arrangement where the order of
elements in the sequence is taken into account) of m trials in a bag containing n different balls
with replacement after each trial. In France this result is traditionally called a p-list.

Draft
Examples:
E1. How many (ordered) words of 7 letters can we form from a separate alpha-
bet of 24 letters (very useful to know the number of trials to find a password for
example)? The solution is:
¯A7
24 = 247
= 4, 586, 471, 424 (4.6.93)
E2. How many groups of people will we have in a referendum on 5 subjects and where
each can be either accepted or rejected? The solution is (widely used in some Swiss
companies):
¯A5
2 = 25
= 32 (4.6.94)
A simple generalization of this result can consist of the following problem statement:
If we have m such objects k1, k2, ..., km as ki may take ni different values then the number of
possible combinations is:
Amn = n1n2...ni...nm (4.6.95)
And if n1 = n2 = ... = nm we have equation then we fall back on:
Amn = Am
n = n1n2...ni...nm = nm
(4.6.96)
4.6.5.2 Simple Permutations without Repetitions
Definition: A simple permutation without repetition (formerly called substitution) of n
distinct objects is an ordered (different) sequence of these n objects all different by definition
in the sequence (without repetition).
Remark
Be careful not to confuse the concept of permutation (n elements between them) and this
of arrangement (of n elements among m)!
The number of permutations of n items can be calculated by induction: there are n places for
a first element, n − 1 for a second element, ..., and there will be only one place for the last
remaining element.
It is therefore trivial that we the number of permutations is given by:
n(n − 1)(n − 2)(n − 3)...(n − (n − 1)) (4.6.97)
Recall that the product:
n(n − 1)(n − 2)(n − 3)...(n − (n − 1)) =
n
i=1
i (4.6.98)

Draft
is called de factorial of n and we note it n! for n ∈ N.
There is therefore for n distinguishable elements:
An =
n
i=1
i = n! (4.6.99)
as possible permutations. This type of calculation can be useful for example in project manage-
ment (calculation of the number of different ways to get in a production line n different parts
ordered from external suppliers).
Example:
How many (ordered) words of 7 different letters without repetition can we cre-
ate?
An = 7! = 5040 (4.6.100)
This result leads us to assimilate it to the ordered results (an arrangement An equation in
which the order of elements in the sequence is taken into account) of the trial of balls that
are all different from a bag containing n distinguishable balls without replacement.
4.6.5.3 Simple Permutations with Repetitions
Deﬁnition: A simple permutation with repetition is when we consider the number of ordered
permutations (different) of a sequence of n distinct objects not necessarily all different in a
given quantity.
Remark
Do not confuse this deﬁnition with the simple arrangement with repetition seen before!
When some elements are not all distinguishable in a sequence of objects (they are repeated in
the sequence), then the number of permutations that we can be do are then trivially reduced to
a smaller number then if all the elements were all distinguishable.
Consider ni as the number of objects of the type i, with:
n1 + n2 + ... + nk = n (4.6.101)
then we write:
¯An(n1, ..., ni, ..., nk) (4.6.102)
the number of possible permutations (yet unknown) with repetition (one or more elements in a
sequence of repetitive elements are not distinguishable by permutation).

Draft
If each of the ni positions occupied by identical elements were occupied by different elements,
the number of permutations could then have to be multiplied by each of the ni! (previous case).
¯An(n1, n2, ..., nk)n1!n2!...nk! = n! (4.6.103)
then we deduce:
¯An(n1, n2, ..., nk) =
n!
n1!n2!...nk!
(4.6.104)
If the n objects are all different in the sequence, we then have:
n1! = n2! = ... = nk! = 1! = 1 (4.6.105)
and we fall back again on a simple permutation (without repetition) as:
¯An(n1, n2, ..., nk) =
n!
n1!n2!...nk!
=
n!
1!1!...1!
= n! (4.6.106)
Example:
How many (ordered) words can we create with the letters of the word Missis-
sippi:
¯A4( 1
M
, 2
PP
, 4
IIII
, 4
SSSS
) =
11!
1!2!4!4!
= 34, 650 (4.6.107)
This result leads us to assimilate it to an ordered result (a permutation ¯An where the order of
elements in the sequence is not taken into account) of the trial of balls that are not all different
from a bag containing k ≥ n balls with limited replacement for each ball.
4.6.5.4 Simple Arrangements with Repetitions
Definition: A simple arrangement without repetition is an ordered sequence of p objects all
distinct taken from n distinct objects with n ≥ p.
We now propose to enumerate the possible arrangements of n objects among p without repeti-
tion. We denote Ap
n the number of these arrangements.
It is easy to calculate that A1
n = n and to check that A2
n = n(n − 1). Indeed, there are n ways
to choose the first object and (n − 1) ways to choose the second when we already have the first.
To determine a nice expression for Ap
n , we reason by induction. We assume equation known
and we deduce that:
Ap
n = Ap−1
n [n − (p − 1)] = Ap−1
n (n − p + 1) (4.6.108)

Draft
It comes:
Ap
n = n(n − 1)(n − 2)(n − 3)...(n − (p − 1)) (4.6.109)
then:
n! = Ap
n(n − p)! = [n(n − 1)(n − 2)...(n − p + 2)(n − p + 1)] (n − p)(n − p − 1)...
(4.6.110)
whence:
Ap
n =
n!
(n − p)! (4.6.111)
This result leads us to assimilate it to the ordered results (an arrangement Ap
n in which the order
of elements in the sequence is taken into account) of the trial of p distinct balls from a bag
containing n different balls without replacement.
Example:
Consider the 24 letters of the alphabet, how many (ordered) words of 7 distinct
letters can we create?
A7
24 =
24!
(24 − 7)!
= 1, 744, 364, 160 (4.6.112)
The reader may have noticed that if p = n we end up with:
Ap
n =
n!
0!
= n! (4.6.113)
So we can say that a simple permutation of n elements without repetition is like a simple ar-
rangement without repetition when n = p.
4.6.5.5 Simple Combinations without Repetitions
Definition: A simple combination without repetitions or choice is an unordered sequence
(where the order doesn’t interest us!) of p elements all different (not necessarily in the visual
sense of the word!) selected from n distinct objects and is by definition noted Cn
p on this website
and named the binomial or binomial coefficient.
If we permute the elements of each simple arrangement of elements p of n, we get all simple
permutations and we know that there are in a number of p!, using the notation convention of
this book we then have (contrary to that recommended one by ISO 31-11!):
Cn
p =
n
p
=
Ap
n
p!
=
n!
p!(n − p)! (4.6.114)

Draft
It is a relation often used in gambling but also in the industry trough the hypergeometric dis-
tribution (see section Quantitative Management as well as quite high level statistics like order
statistics (see chapter Statistics).
Remark
R1. We have necessarily by construction Cn
p ≤ Ap
n.
R2. Depending on the authors we inverse the index and sufﬁx of C then you
must be careful!
This result leads us to assimilate it to the unordered result (an arrangement Cp
n in which the
order of elements of the sequence is not taken into account) of the trial of p balls of a bag
containing n different balls without replacement.
Example:
Consider the 24 letters of the alphabet, how many choices do we have to take 7
letters in the 24 without taking into account the trial order?
C24
7 =
24!
7!(24 − 7)!
= 346, 104 (4.6.115)
The same value can be obtained with the function gywfsx@ A of Microsoft Excel 11.8346
(English version).
There is, in relation to the binomial coefﬁcient another relation very often used in many case
studies and also more globally in physics or functional analysis. This is the Pascal’s Formula:
Proof 4.21.2.
Cn−1
p−1 + Cn−1
p =
(n − 1)!
((n − 1) − (p − 1))!(p − 1)!
+
(n − 1)!
(n − 1 − p)!p!
=
(n − 1)!
(n − p)!(p − 1)!
+
(n − 1)!
(n − 1 − p)!p!
(4.6.116)
We also have p! = p(p − 1)!, then:
(p − 1)! =
p!
p
(4.6.117)
and because (n − p)(n − p − 1)! = (n − p)!:
(n − p − 1)! =
(n − p)!
n − p
(4.6.118)

Draft
Then:
Cn−1
p−1 + Cn−1
p =
(n − 1)!
(n − p)!(p − 1)!
+
(n − 1)!
(n − 1 − p)!p!
=
(n − 1)!p
(n − p)!p!
+
(n − 1)!(n − p)
(n − p)!p!
=
(n − 1)!
(n − p)!p!
[p + (n − p)] =
(n − 1)!n
(n − p)!p!
=
n!
(n − p)!p!
= Cn
p
(4.6.119)
Q.E.D.
4.6.5.6 Simple Combinations with Repetitions
Definition: A simple combination with repetition of p elements of n is a collection of p
unordered elements, not necessarily distinct.
Simple combinations with repetition are very important for the Wald-Wolfowitz statistical test
used in economics and biology and that we will study in the Statistics section.
We will Introduce this kind of combination directly with an example an ingenious approach that
we have thanks to the physicist and 1938 Nobel Prize in Physics: Enrico Fermi.
Consider {a, b, c, d, e, f} a set having a number n of elements equal to 6 and where we draw a
number of elements p equal to 8. We would like to calculate the number of combinations with
repetition of elements in a starting set of cardinal 6 in a destination set of cardinal 8.
Consider, for example, the following three combinations:
a a b b b e e f (4.6.120)
b b d d d e e e (4.6.121)
b b b d d d d d (4.6.122)
where as the order of elements does not occur, we have grouped the elements to facilitate the
reading. Now represent all the above elements by the same symbol 0 and separate the groups
consisting of a single element by bars (this is the trick Enrico Fermi). Thus, when one or more
elements are not included in a combination, we still denote the separation bars (corresponding
to the number of missing elements + the separation of group). Thus, the three combinations
above can be written as:
0 0 | 0 0 0 | | | 0 0 | 0 (4.6.123)
| 0 0 | | 0 0 0 | 0 0 0 | (4.6.124)
| 0 0 0 0 | | 0 0 0 0 | | (4.6.125)
We see above that in each case, there are eight 0 (logic. ..) but also that there are also always
five |. The number of combinations with repetitions of six elements of a starting set to the
final one of 8 elements is equal to the number of permutations with repetitions of 8 + 5 = 13
elements, so:
13!
5!8!
(4.6.126)

Draft
We also see that in the general case the number of combinations without consideration of repe-
titions order can also be written:
(n + p − 1)!
(n − 1)!p!
(4.6.127)
It is traditional to write this:
Γn
p =
(n + p − 1)!
(n − 1)!p!
(4.6.128)
We also see that:
Γn
p =
(n + p − 1)!
(n − 1)!p!
=
(n + p − 1)!
((n + p − 1) − p)!p!
= Cn+p−1
p (4.6.129)
Then:
Γn
p = Cn+p−1
p =
(n + p − 1)!
(n − 1)!p! (4.6.130)
That we also sometimes write:
Γn
p = Cn+p−1
p = Cn+p−1
n−1 =
(n + p − 1)!
(n − 1)!p!
(4.6.131)
To resume:
Type Expression
Simple arrangement with repetitions ¯Am
n = nm
Simple arrangement without repetitions An
m =
n
(n − m)!
Simple permutation without repetitions An = n!
Simple permutation with repetitions ¯An(n1, n2, ..., nk) =
n!
n1!n2!...nk!
Simple combination without repetitions (case
of the simple arrangement without repetitions
where the order is not taken into account)
Cn
m =
n
p
=
Am
n
m!
=
n!
m!(n − m)!
Simple combination with repetitions (case of
the simple permutation with repetition where
the order is not taken into account)
Γn
p = Cn+p−1
p =
(n + p − 1)!
(n − 1)!p!

Draft
4.6.6 Markov Chains
Markov chains are simple but powerful probabilistic and statistical tools but for which the
choice of the mathematical presentation can sometimes be a nightmare... We will try here
to simplify a maximum the notation to introduce this great tool widely used in businesses to
manage logistics, queues for call centers or cash desk to the theory of failure for preventive
maintenance, statistical physics and biological engineering (and the list goes on and for more
details the reader should refer to the relevant chapters available on this website...).
Definitions:
D1. We note by {X(t)}t∈T a probabilistic process function of time whose value at any time
depends on the outcome of a random experiment. Thus, at each time t, X(t) is a random
variable that we name stochastic process (for more details on financial applications see
the chapter Economy).
D2. If we consider a discrete time, we then note discrete time stochastic process as
{Xn}n∈N.
D3. If we further assume that the random variables Xn can take only a discrete set of values
we then speak of process in discrete time and discrete space.
Remark
It is quite possible as in the study of communications flows (see section Quantitative
Management) of having a process in continuous time and discrete state space.
Definition: {Xn}n∈N is a Markov chain if and only if:
P(Xn = j | Xn−1 = in−2, Xn−2 = in−2, ..., X0 = i0) = P(Xn = j | Xn−1 = in−1) (4.6.132)

Draft
in other words (it is very easy!) the probability that the chain is in a certain state on the n-th step
of the process depends only on the state of the process at step n − 1 and not on any previous
steps!
Remark
Also in probabilities a stochastic process verifies the Markov property if and only if the
conditional probability distribution of future states, given the present moment, depends
only on the present state and not even past states as the relation above. A process with
this property is also called a Markov process.
Definition: A homogeneous Markov chain is a chain such that the probability that it has to
go in a certain state at the n-th stage is independent of time. In other words, the probability
distribution characterizing the next step does not depend on time (of the previous step), at all
times the probability distribution of the chain is always the same for characterizing the transition
to the current step.
We can then define (reduce) the law of probability transition of a state i to state j by:
pij = P(Xn = j | Xn−1 = i) (4.6.133)
It is then natural to define the transition matrix or stochastic matrix:
M =





p11 p12 · · · p1n
p21 p22 · · · p2n
...
...
...
...
pm1 pm2 · · · pmn





(4.6.134)
as the matrix that contains all possible transition probabilities between states in an oriented
graph.
Markov chains can be represented graphically as an oriented graph G (see section Graph The-
ory) sometimes named automate having for the top points (states) i and for the edges the
oriented couples (i, j). We then associate to each component an oriented arc and a transition
probability.

Draft
Example:
Figure 4.60 – Generic example of a Markov chain
Thus, in the example of the oriented graph above, the only allowed transitions from the
above 4 states (4 × 4 matrix) are those indicated by the arrows. So that the transition
matrix is then simpliﬁed to:
P =




p11 p12 p13 p14
p21 p22 p23 p24
p31 p32 p33 p34
p41 p42 p43 p44



 =




0 p12 0 p14
0 0 p23 0
p31 0 p33 0
p41 0 0 0



 (4.6.135)
The reader has seen in the previous example that we have the trivial property (by construction!)
that the sum of the terms (probabilities) of a row of the matrix P is always unitary (and therefore
the sum of the terms of a column of the transpose of the matrix unit is still equal to the unit too):
j
pij = 1 (4.6.136)
and that the matrix is positive (meaning that all its terms are non-negative).
Remark
Remember that the sum of the probabilities of the columns is always equal to 1 for the
transpose of the stochastic matrix!
The analysis of transient state (or: random walk) of a Markov chain consist to determine (or to

Draft
impose!) to the column-matrix (vector) p(n) to be in a given state at n-th step of the walk:
p(n) =





p1(n)
p2(n)
...
pCard(E)(n)





(4.6.137)
with the sum of the components that is always equal 1 (since the sum of the probabilities of
being in any of the vertices of the graph at given a time/step must be equal to 100%).
We frequently name this column matrix stochastic vector or probability measure on the ver-
tex i.
Theorem 4.22. We want to prove that the total probability of this stochastic vector is always
equal to 1.
Proof 4.22.1. If p(n) is a stochastic vector, then its image:
p(n + 1) = PT
p(n) (4.6.138)
is also a stochastic vector. Effectively, pi(n + 1) ≥ 0 because:
pi(n + 1) =
i
pijpj(n) (4.6.139)
is a sum of positive or zero values. Furthermore, we ﬁnd:
pi(n + 1) =
i j
pijpj(n) =
j i
pijpj(n) =
j i
pij pj(n)
=
j
1 · pj(n) =
j
1pj(n) = 1
(4.6.140)
Q.E.D.
This probability vector whose components are positive or zero, depends (it’s pretty intuitive) on
the transition matrix P and the vector of initial probabilities p(0).
Although it is provable (Perron-Frobenius theorem), the reader may verify by a practical case
(computerized or not!) that if we choose any vector state p(n) then there exists for any stochastic
matrix P a unique probability vector traditionally noted π as:
PT
π = π (4.6.141)
Such a probability measure π satisfying the above equation is called an invariant measure or
stationary measure or balance measure which represents the equilibrium state of the system.
In terms of linear algebra (see section of the same name) for the eigenvalue 1, is an eigenvector
of P (see section Linear Algebra).
We will see a trivial example in the Graph Theory section which will be redeveloped in detailed
as in the section of Game and Decision Theory in the context of pharmaco-economics and in

Draft
the section of Software Engineering when we will study the fundamentals of the Google Page
Rank algorithm. But also note that the Markov chains are used for example in meteorology (or
in the case of computer passwords hacks):
Figure 4.61 – Concrete example of a very simple Markov chain
or in medicine, finance, transportation, marketing, etc.
In the field of language analysis, from the frequency analysis of a sequence of words, computers
are able to also build Markov chains and therefore propose a more correct semantic during
grammatical computerized corrections or in written transcription of oral presentations.
Definitions:
D1. A Markov chain is said to be an irreducible Markov chain if all states are bound to
others (it’s the case of the example in the figure above).
D2. A Markov chain is said to be an absorbing Markov chain if one of the states of the chain
absorbs the transitions (so nothing comes out just to say things in a more simple way!).
∼90 %
27 votes, 51.11%

Draft
4.7 Statistics
S
Tatistics is a science that concerns the systematic grouping of facts or recurring events
that lend themselves to a numerical or qualitative assessment over time according to a
given law. In the industry and the economy in general, statistics is a science that helps
in an uncertain environment to make valid inferences.
You should know that among all areas of mathematics, the one that is used the most widely in
business and research centres is Statistics and especially since softwares greatly facilitates the
calculations! This is why this chapter is one of the biggest on this internet website even if only
the basic concepts are presented!
Note also that Statistics have a very bad reputation at the university because the notations are
often confusing and vary greatly from one teacher to another, from one book to another, from
one practitioner to another. Strictly speaking, it should comply with the vocabulary and notation
of the ISO 3534-1:2006 norm and unfortunately this chapter was written before the publication
of this standard ... a certain period of adaptation will be necessary to obtain the full compliance.
It is perhaps useless to precise that Statistic is widely used in engineering, theoretical physics,
fundamental physics, econometrics, project management and in the industry of process, in the
fields of life and non-life insurance, in the actuarial science or in the database analysis (with
Microsoft Excel very often ... unfortunately ....) and the list goes on. We will also meet quite
often the tools presented here in the chapters of Fluid Mechanics, Thermodynamics, Technical
Management, Industrial Engineering and Economy (especially in the last two). The reader can
then refer to them to have some concrete practical applications of the most important theoretical
elements that will be seen here.
Note also that in addition to a few simple examples on these pages, many other application ex-
amples are given on the exercise server of this website in the categories Probability and Statis-
tics, Industrial Engineering, Econometrics and Management Techniques.
Definition: The main purpose of Statistics is to determine the characteristics of a given popula-
tion from the study of a part of the population, called sample or representative sample. The
determination of these characteristics should enable statistics to be a tool for the decision help!
Remark
The data processing concerns the descriptive statistics. The interpretation of data from
estimators is called statistical inference (or inferential statistics), and mass data anal-
ysis statistical frequency as opposed to Bayesian inference (see section Probabilities).
When we observe an event taking into account some given factors, there can happen that a sec-
ond observation takes place in conditions that seem identical. By repeating these steps several
times on different supposedly similar objects, we find that the observed results are statistically
distributed around a mean value that is ultimately the most likely possible outcome. In prac-
tice, however, we sometimes perform a single measurement and then the goal is determine the
value of the error we make by adopting it as average measure. This determination requires
knowledge of the type of statistical distribution we are dealing with and that is on what we will

Draft
focus (among others) to study here (at least the basics!). However, there are several common
methodological approaches whe we face the hazard (less common are not mentioned yet):
1. A first is to simply ignore the random elements, for the simple reason that we do not know
how to integrate them. We then use the scenarios method also called deterministic
simulation. This is typically the tool used by financial managers and non-graduates
managers with tools like Microsoft Excel (which includes a scenarios management tool)
or MS Project (which includes a tool to manage the deterministic optimistic, pessimistic
and expected scenarios).
2. A second possible approach, when we do not know how to associate probabilities to
specific future random events, is game theory (see section Game and Decision Theory)
where semi-empirical criteria of selection are used as the criterion of maximax, minimax,
Laplace, Savage, etc.
3. Finally, when we can link probabilities to random events, whether these probabilities
derived from calculations or measurements, whether they are based on experience from
previous similar situations as the current situation, we can use descriptive and inferential
statistics (contents of this chapter) to obtain usable and relevant information from this
mass of acquired datas.
4. A last approach when we know the relative probabilities from intervening events in re-
sponse to strategic choices is the use of decision theory (see section Game and Decision
Theory).
Remarks
R1. Without mathematical statistics, a calculation on datas (e.g. an average), is a
ponctual indicator. This is mathematical statistics which gives it the status of estimator
whose bias, uncertainty and other statistical characteristics are controlled. We generally
seek to ensure that the estimator is unbiased, convergent and efficient (we will see during
our study of estimators further what is exactly all that stuff).
R2. When we communicate a statistic it should be an obligation to specify the
confidence interval, the p-value and the size of the studied sample (absolute statistics)
and its detailed characteristics and make available the sources and data protocol
otherwise it has almost no scientific value (we will see all these concepts in detail further
below). A common mistake is to communicate in relative values. For example, on a test
group of 1,000 women, 5 women will die from breast cancer without screening check,
with screening check 4 women will die. Some will say a little to quickly (typically
physicians....) that screening checks saves 20% of women (relative value as one the
of five could have been saved...). In fact this is wrong since the absolute benefit of
screening is insignificant!
R3. If you have a teacher or trainer who dares to teach you statistics and proba-
bilities only with examples based on gambling (cards, dice, matches, coins, etc.) dispose
or denounce him. Normally examples should be based on the industry, economy or
RD, i.e. in areas used in daily by businesses!).

Draft
4.7.1 Samples
During the statistical analysis of sets of information, the way to select the sample is as important
as how to analyze it.The sample must be representative of the population (we do not necessarily
make reference to human populations!). For this, the random sampling is the best way to achieve
it.
Deﬁnitions:
D1. The statistician always starts from the observation of a ﬁnite number of elements, which
we name the population. The observed elements, in quantity n, are all of the same
nature, but this nature can be very different from one population to another.
D2. We are in the presence of a quantitative character when each observed element is ex-
plicitly subject to the same measure. To a given quantitative character, we associate a
quantitative variable continuous or discrete, which summarizes all the possible val-
ues that the measure can take (such information being represented by functions like the
Gauss-Laplace distribution, the beta distribution, the Poisson distribution, etc.).
Remark
We will come back on the concept of variable and distribution a little further...
D3. We are in the presence of a qualitative character when each observed element is ex-
plicitly subject to a single connection to a modality from a set of exclusive modalities
(e.g.: man | woman) that permits to classify all studied elements in a given certain point
of view (such information being represented by bar charts, sector charts, bubble charts,
etc.). All modalities of a character can be established a priori before the survey (a list,
a nomenclature, a code) or after the survey. A study population can be represented by a
mixed character, or set of modalities such as gender, wage range, age, number of children,
marital status for example for an individual.
D4. A random sample is by default (without more precision) a sample in which all indi-
viduals in a population have the same chance, or equally likely probability (and we
emphasize that this probability must be equal), to end up in the sample.
D5. In the opposite in a sample whose elements were not chosen randomly, then we talk about
a biased sample (in the opposite case we talk about a non-biased sample).
Remark
A small representative sample is by far preferable to a large biased sample. But
when the sample sizes are small, the hazard can a result worst than the biased
one...

Draft
4.7.2 Averages
The concept of average or central tendency (financial analysts call it a measure of loca-
tion...) is with the notion of variable at the basis of statistics.
This notion seems very familiar to us and we talk a lot about it without asking too many ques-
tions. But there are various qualifiers (we emphasize that this are only qualifiers!) to distinguish
the way of the resolution of a problem of calculating the average.
Thus, you must be very very careful about the calculations of averages because there is a ten-
dency in business to rush and to systematically use the arithmetic mean without thinking, which
can lead to serious errors! A nice example (for an analogy) is that a considerable number of laws
require only moderate levels of pollution per year, while for example, smoking one cigarette per
day during 365 days does not have the same impact as smoking 365 cigarettes in one day dur-
ing one year when both have the same average taken over a year ... This is a clear evidence of
statistical incompetence of the legislature.
Here is a small sample of common mistakes:
• Consider that the arithmetic mean is the value that divides the population into two equal
parts (although it is the median that does this).
• Consider that the average of the ratios of the type goals/realisations is equal to the ratio of
the average of the goals and of the average of the realisations (although it is not the same
thing!).
• Consider that the average salaries of different subsidiaries, is equal to the global average
(while this is true if and only if there is the same number of employees in each subsidiary
of the company).
• Consider that the average of the average of the rows in a table is always equal to the
average of the columns of the same table (although this is true if and only if the cell
contents are not empty).
• Calculate the arithmetic average growth of the revenue in % (as the geometric mean must
be used).
• etc.
We will see below different average with examples relative to arithmetic, to the enumeration,
to physics, to econometrics, to geometry and sociology. The reader will find other practical
examples by browsing the entire book.

Draft
Definitions: As given xi real numbers, then we have:
D1. The arithmetic average or sample average (the most commonly known) is defined as
the quotient of the sum of n observed xi values by the total size n of the sample:
µa =
1
n
n
i=1
xi (4.7.1)
and is very often written ¯x or µ and is for any discrete or continuous statistical distribution
an unbiased estimator of the mean.
The arithmetic average represents a statistical measure expressing the magnitude that
would have each member of a set of measures if the total must be equal to the product of
the arithmetic average by the number of members.
If some values repeats more than once in the measurements, the arithmetic mean is then
often formally noted as following:
µa =
1
n
n
i=1
nixi =
n
i=1
ni
n
xi =
n
i=1
wixi (4.7.2)
and is named weighted average. Finally, we could indicate that under this approach, the
actual weighted average will be named mathematical mean or just mean in the field
of study of probabilities.
We may as well use the frequencies of occurrence of the observed values named classes
frequencies:
fi =
ni
n
(4.7.3)
So that we get another equivalent definition named the weighted average by the classes
frequencies:
µa =
1
n
n
i=1
nixi =
n
i=1
ni
n
xi =
n
i=1
fixi (4.7.4)
Before continuing, it’s important to know that in the field of statistics it is useful and often
necessary to combine measurements/data in class intervals of a given width (see examples
below). We often have to make several tries to choose the intervals even if there are semi-
empirical formulas for choosing the number of classes when we have n available values.
One of these semi-empirical rules used by many practitioners is to retain the smallest
integer k of classes such as:
2k
≥ n (4.7.5)
the width of the class interval is then obtained by dividing the range (difference between
the maximum and minimum measured value) by k. By convention and rigorously... (so
rarely respected in the notations), a class interval is closed on the left and open on the
right (see section Functional Analysis):
[..., ...[ (4.7.6)

Draft
This empirical rule is called the Sturges rule and is based on the following reasoning:
We assume that the values of the binomial coefficient Ci
k gives the number of individuals
in an ideal histogram (we let the reader check this simply with a spreadsheet software
like Microsoft Excel 11.8346 and the gywfsx@kDiA function) of k intervals for the i-th
interval. As as k becomes large the histogram looks more and more like a continuous
curve called the Normal curve or bell curve as as we will see later.
Therefore, based on the binomial theorem (see section Calculus), we have:
n =
k
i=0
Ci
k =
k
i=0
Ci
kai−k
bk
=
k
i=0
Ci
k1i−k
1k
=
k
i=0
Ci
k = (a + b)k
= (1 + 1)k
= 2k
(4.7.7)
Then, for each interval i the practitioner will traditionally take the average between the
lower and upper limit for the calculation and multiply it by the corresponding class fre-
quency fi. Therefore, the grouping of class frequencies implies that:
(a) The weighted average by the frequencies differs from the arithmetic average.
(b) As the approximation seen above it will be a worst indicator compare to the arith-
metic average...
(c) It is very sensitive to the choice of the number of classes, than very bad at this level.
There are many other empirical rules for the discretization of random variables. For
example, the software XLStat offers not less than 10 rules (constant amplitude, Fisher
algorithm, k-means, 20/80, etc.).
Later, we will see two very important properties of the arithmetic average and of the mean
that you will have to understand absolutely (the weighted average of deviations from the
average and the average deviations from the average).
Remark
The mode, noted Mod or simply M0, is defined as the value that appears most
often in a set of data. In Microsoft Excel 11.8346, it is important to know that the
wyhi@ A function returns the first value in the order of values having the largest
number of occurrences therefore assuming a unimodal distribution.
D2. The median or middle value, noted Me is the value that cut the population values
into two equal parts. In the case of a continuous statistical distribution f(x) of a random
variable X, it is the value that represents the value that has 50% of cumulative probability
to occur (we will see further in details the concept of statistical distribution):
P(X ≤ Me) = P(X ≥ Me) =
Meˆ
−∞
f(x)dx =
+∞ˆ
Me
f(x)dx = 0.5 (4.7.8)
In the case of a series of ordered values x1, x2, ..., xi, ...xn, the median is therefore by
definition the value such that we have the same number of values that are greater than or
equal to it than the number of values that are less or equal to it.

Draft
Remarks
R1. The median is mainly used for skewed distributions, because it represent them
better than the arithmetic average.
R2. The median is in practice often not a single value (at least in the case
where n is even). Indeed, between the values corresponding to ranges
n
2
and
n
2
+1
there is an infinite number of values to choose which cut the population in half.
More rigorously:
• If the number of terms is odd, i.e. of the form 2n + 1, the median of the series is the
term of order n + 1 (that the terms are all distinct or not!).
• If the number of terms is even, i.e. of the form 2n, the median of the series is half-
sum (arithmetic average) of the values of the terms of rank n and n + 1 (that the
terms are all distinct or not!).
In any case, by this definition, it follows that there are at least 50% of the terms of the
serie that are smaller than or equal to the median, and at least 50% of the terms of the
serie that are greater than or equal to the median.
For example, consider the table of wages below:
Employee N˚ Wage Cumulated Employees %Cumulated Employees
1 1,200 1 6%
2 1,220 2 12%
3 1,250 3 18%
4 1,300 4 24%
5 1,350 5 29%
6 1,450 6 35%
7 1,450 7 41%
8 1,560 8 47%
9 1,600 9 53%
10 1,800 10 59%
11 1,900 11 65%
12 2,150 12 71%
13 2,310 13 76%
14 2,610 14 82%
15 3,000 15 88%
16 3,400 16 94%
17 4,800 17 100%
Table 4.16 – Identification of the median
There is in the table an odd number 2n + 1 of values. So the median of the series is the
term of rank n + 1. This is 1, 600.− (result that give any spreadsheet software). The
arithmetic average is in this case about 2, 020.−.
In direct relation with the median it is important to define the following concept to under-
stand the underlying mechanism:

Draft
Definition: Be given a statistical series x1, x2, ..., xi, ..., xn, we name dispersion of ab-
solute differences around x the number ε (x) defined by:
ε (x) =
i
| xi − x | (4.7.9)
ε (x) is minimum for a value of x closest to a given value xi in the sense of the absolute
error value. The median is the value that achieves this minimum (extremum)! The idea
will then be to study the variations of the function to find the position of this extremum.
Indeed, we can write:
∀x ∈ [xr, xr+1] , r ∈ {1, 2, 3, ..., n − 1} ε (x) =
r
i=1
| xi − x | +
n
i=r+1
| xi − x |
(4.7.10)
Then by definition of the x value:
ε (x) =
r
i=1
| xi − x | +
n
i=r+1
| xi − x |
= [rx − (x1 + x2 + ... + xr) − [(xr+1 + ... + xn − (n − r)x]]
= (2r − n)x + (xr+1 + ... + xn) − (x1 + x2 + ... + xr)
(4.7.11)
What allows us to skip the absolute values is simply the choice of the index r that is taken
so that the serie of values in practice can always be split into two parts: all that is less than
the element indexed by r and all that is superior to it (i.e.. the median by anticipation...).
ε (x) is also a piecewise (discrete) affine function (similar to the equation of a line for
fixed fixed values of r and n) where we see that by analogy the factor:
2r − n (4.7.12)
is the slope of the function and:
(xr+1 + ... + xn) − (x1 + x2 + ... + xr) (4.7.13)
the Y -intercept (ordinates at the origin).
The function is decreasing (negative slope) until r is less than
n
2
and increasing when
r is greater than
n
2
(it passes trough an extremum!). Specifically, we distinguish two
particularly cases of interest since n is an integer:
• If n is even, we can say that n = 2n , then the slope can be written 2(r − n ) and
it is equal to zero if r = n and then, as the result is valid by construction only
for ∀x ∈ [xr, xr+1] then ε (x) is constant on [xn , xn +1] and we have an extremum
necessarily in the middle of this range (arithmetic average of the two terms).
• If n is odd, we can say that n = 2n + 1 (we cut the series into two equal parts),
then the slope can be written (2r − 2n − 1) and it is zero if r = n +
1
2
, as the result
is only valid for ∀x ∈ [xr, xr+1] then it is immediate that the middle value is the
median xn +1.

Draft
We ﬁnd out the median in both cases. We will also see later how the median is deﬁned
for a continuous random variable (the underlying idea is the same).
There is another practical case where the statistician has at its disposal only the values
grouped in intervals of statistical classes. The procedure for determining the median is
then different:
When we have at our disposal only a values grouped in intervals of statistical classes, the
abscissa of the point of the median is generally within a class. To then get a more accurate
value of the median, we perform a linear interpolation. This is what we name the linear
interpolation method of the median.
The median value can be read from the graph or calculated analytically. Indeed, consider
the graph of the cumulative probability F(x) in class intervals as below where the bounds
of the intervals were connected by straight lines:
Figure 4.62 – Graphical representation of the estimation of the median by linear interpolation
The value of the median Me is obviously located at the crossroads between the cumulated
probability of 50% (0.5) and the abscissa. Thus, by applying the basics of functional
analysis, we have (just by observing that the slope in the interval containing the median
is equal in the half-interval to the left and to right adjacent to the median):
∆x
∆y
=
Me − 2
0.5 − 0.2
=
4 − 2
0.7 − 0.2
(4.7.14)
What we frequently write:
Me − a
0.5 − F(a)
=
b − a
F(b) − F(a)
(4.7.15)
Thus the value of the median:
Me = a + (b − a)
0.5 − F(a)
F(b) − F(a)
(4.7.16)
Consider the following table that we will see again much later in this chapter:

Draft
Price Number Cumulated number Relative frequencies
of tickets of tickets of tickets of tickets
[0,50[ 668 668 0.068
[50,100[ 919 1,587 0.1587
[100,150[ 1,498 3,085 0.3085
[150,200[ 1,915 5,000 0.5000
[200,250[ 1,915 6,915 0.6915
[250,300[ 1,498 8,413 0.8413
[300,350[ 919 9,332 0.9332
[350,400[ 440 9,772 0.9772
[400 and + 228 10,000 1
Table 4.17 – Identiﬁcation of the median and the mode
We see that the median class is in the range [150, 200] because the cumulative value
of 0.5 is there (column at the right of the table) but the median has, using the previously
established relation, the precise value of (it is trivial in the particular example of the table
above, but we still do the calculation...):
Me = 150 + (200 − 150)
0.5 − 0.3805
0.5 − 0.3805
= 200 (4.7.17)
and of course we can do the same with any other percentile!
We can also give a deﬁnition to determine the modal value if we are only in possession
of the frequencies of class intervals. To see that we start with diagram below named
grouped distribution in frequencies bar:
Figure 4.63 – Graphical representation of the estimation of the modal value with classes intervals
Using Thales relations (see section Euclidean Geometry), it comes immediately, noting
M the modal value:
M − xi
∆1
=
xi+1 − M
∆2
(4.7.18)

Draft
As in a proportion, we do not change the value of the ratio by adding the numerators and
adding the denominators, we get:
M − xi
∆1
=
xi+1 − M
∆2
=
xi+1 − xi
∆1 + ∆2
(4.7.19)
We then have:
M = xi +
∆1
∆1 + ∆2
(xi+1 − xi) (4.7.20)
With the previous example this gives then:
M = 150 +
(1, 915 − 1, 498)
(1, 915 − 1, 498) + (1, 915 − 1, 498)
(250 − 150)
= 150 +
1
2
(250 − 150) = 150 +
1
2
100 = 200
(4.7.21)
The question that then arises is to the appropriateness of the choice of the mean, mode
or median in terms of communication ... (normally we communicate them all three in
corporate reports!).
A good example is that of the labor market where in general, while the average wage and
the median wage are quite different, the institutions of state statistics calculate the median
than many traditional media then explicitly equate to he concept of arithmetic average
in their news...
Remark
To avoid getting an arithmetic average having little sense, we often calculate a
trimmed average, i.e. an arithmetic average calculated after removing outliers in
the series (using Grubbs or Dixon Tests).
The quantile generalize the notion median by cutting the distribution in sets of equal
parts (of the same cardinality we might say ...) or in other words in regular intervals.
We define the quartiles, the decile and percentile on the population, ordered in
ascending order, that we divide by 4, 10 or 100 parts of the same size.
So we talk about the 90th percentile to indicate the value separating the first 90% of the
population and the 10% remaining.
Note that in Microsoft Excel 11.8346 the functions …e‚„svi@ A, €i‚gix„svi@ A,
wihsex@ A, €i‚gix„‚exu@ A are available and it can be useful that we specify that
there are several variants of calculating these percentiles that explains possible variation
between the results of different spreadsheet softwares.
This concept is very important in the context of confidence intervals that we will see much
further in this section and very useful in the field of quality with the use of box plots
(also named Box Whiskers plots) to compare (discriminate as experts say) quickly
two populations of data or more and especially to eliminate outliers (taking as reference
the median will just make more sense!).

Draft
Figure 4.64 – Box Whiskers Plot
Another very important mental representation of box plots is the following (it can get an
idea of the asymmetry of the distribution as is able to do the R software):

Draft
Figure 4.65 – Graphical representation of the mode, median and 1st + 3rd quartile compared to a distribution
The concepts of median, outliers and confidence intervals that have yet been proved
and/or just defined are so significant that there exists international standards for their
proper use. First let us cite the norm ISO 16269-7:2001 Median - Estimation and confi-
dence intervals and also the norm ISO 16269-4:2010 Detection and treatment of outliers.
D3. By analogy with the median, we define the medial as the value (in ascending order of
values) that shares the (cumulative) sum values into two equal masses (i.e. the total sum
divided by two).
In the case of our wages example, while the median gives the 50% of the salaries being
below and above the medial gives how many employees share (and therefore the sharing
wage) the first half and how many employees share the second half of the total of the
wages costs.

Draft
Employee N˚ Wage Cumulated Wages %Cumulated Wages
1 1,200 1,200 3.5%
2 1,220 2,420 7%
3 1,250 3,670 10.7%
4 1,300 4,970 14.5%
5 1,350 6,320 18.4%
6 1,450 7,770 22.6%
7 1,450 9,220 26.8%
8 1,560 10,780 31.4%
9 1,600 12,380 36.1%
10 1,800 14,180 41.3%
11 1,900 16,080 46.8%
12 2,150 18,230 53.1%
13 2,310 20,540 59.8%
14 2,610 23,140 67.4%
15 3,000 26,140 76.1%
16 3,400 29,540 86%
17 4,800 34,340 100%
Table 4.18 – Identiﬁcation of the mediale
The sum of all wages is equal to 34, 340 and therefore the medial is 17, 170 then the
medial is between the employees No. 11 and 12, while the median was 1, 600. We see
then that the medial corresponds to 50% of the aggregate. This is a very useful indicator
in Pareto or Lorenz analysis (see section Quantitative Management).
D4. The root mean square sometimes denoted simply Q which comes from the general
relation:
µq = m
n
i=1
xm
i
n
(4.7.22)
but where we take m = 2.
Example:
Consider a square of side a, and another square of side b. The average area
of two squares equals one square of side:
µ2
q =
a2
+ b2
2
⇒ µq =
a2
+ b2
2
(4.7.23)
In Microsoft Excel 11.8346 you can combine the functions ƒ…wƒ@ A, gy…x„@ A and to
quickly calculate the root mean square as following:
a@ƒ…wƒ@FFFAGgy…x„@FFFAA”@IGgy…x„@FFFAA

Draft
D5. The harmonic mean sometimes simply denoted H is defined by:
1
µh
=
n
i=1
x−1
i
n
(4.7.24)
It is little known but is often the result of simple and relevant arguments (typically the
equivalent resistance of an electrical circuit with several resistors in parallel). There is a
re‚wiex@ A function in Microsoft Excel 11.8346 to calculate it.
Example:
Consider a distance d travelled in one direction at the speed v1 and in the
other direction (or not) at the speed v2. The arithmetic average speed will be
obtained by dividing the total distance 2d by the time of the travel:
v =
2d
t
(4.7.25)
If we calculate the time it takes when travel d with a speed vi that is simply the
quotient:
ti =
d
vi
(4.7.26)
The total time is therefore:
t =
2d
v
= t1 + t2 =
d
v1
+
d
v2
(4.7.27)
If the distance is not the same for the both velocities anyway each velocity remains
the same this is why d disappears!
In other words: We use the harmonic mean when are given to us ratios.
D6. The geometric mean sometimes simply denoted G is defined in the general case by:
µg = n
n
i=1
xi (4.7.28)
This average is often forgotten by undergraduate employees but famous it is famous in the
field of finance (see section Economy) this is also why there is an qiywiex@ A function
in Microsoft Excel 11.8346 to calculate it.
A geometric mean is often used when comparing different items – finding a single fig-
ure of merit for these items – when each item has multiple properties that have different
numeric ranges. For example, the geometric mean can give a meaningful average to
compare two companies which are each rated at 0 to 5 for their environmental sustain-
ability, and are rated at 0 to 100 for their financial viability. If an arithmetic mean were
used instead of a geometric mean, the financial viability is given more weight because

Draft
its numeric range is larger so a small percentage change in the financial rating (e.g. go-
ing from 80 to 90) makes a much larger difference in the arithmetic mean than a large
percentage change in environmental sustainability (e.g. going from 2 to 5). The use of a
geometric mean normalizes the ranges being averaged, so that no range dominates the
weighting, and a given percentage change in any of the properties has the same effect on
the geometric mean. So, a 20% change in environmental sustainability from 4 to 4.8 has
the same effect on the geometric mean as a 20% change in financial viability from 60 to
72.
Remark
In 2010, the geometric mean was introduced to compute the Human Develop-
ment Index by United Nations Development Programme. Poor performance in
any dimension is directly reflected in the geometric mean. That is to say, a low
achievement in one dimension is not anymore linearly compensated for by high
achievement in another dimension. The geometric mean reduces the level of sub-
stitutability between dimensions and at the same time ensures that a 1% decline in
index of, say, life expectancy has the same impact on the HDI as a 1% decline in
education or income index. Thus, as a basis for comparisons of achievements, this
method is also more respectful of the intrinsic differences across the dimensions
than a simple average.
Like for the number 0, it is impossible to calculate the geometric mean with negative
numbers. However, there are several work-arounds for this problem, all of which require
that the negative values be converted or transformed to a meaningful positive equivalent
value. Most often this problem arises when it is desired to calculate the geometric mean of
a percent change in a population or a financial return, which includes negative numbers.
For example, to calculate the geometric mean of the values +12%, −8%, 0% and +2%, in-
stead calculate the geometric mean of their decimal multiplier equivalents of 1.12, 0.92, 1
and 1.02, to compute a geometric mean of 1.0125. Subtracting 1 from this value gives the
geometric mean of +1.25% as a net rate of growth (or in financial circles is named the
Compound Annual Growth Rate C.A.G.R.).

Draft
Example:
Suppose that a bank offers an investment opportunity and plans for the first
year an interest (this is absurd, but this is an example) with a rate (X − Y )% but
for the second year with an interest rate (X +Y )%. At the same time another bank
provides a constant interest rate for two years: X%. We will say a little bit to fast
that this is the same... In fact the two investments do not have the same profitability!
In the first bank, a capital C0 will give after the first year of interest:
(X − Y )% · C0 (4.7.29)
and the second year:
(X + Y )% [(X − Y )% · C0] (4.7.30)
In the other bank we will have after one year:
X% · C0 (4.7.31)
and after the second year:
X%(X% · C0) (4.7.32)
and so on...
As you can probably see it the placement will not be identical if Y = 0! X% is the
not the arithmetic average of (X − Y )% and (X + Y )%.
Now if we write:
r1 = (X + Y )% and r2 = (X − Y )% (4.7.33)
What is in reality the average value of the global interest rate r?
After 2 years (for example), the capital is multiplied by r1 · r2. If an average exists
it will be denoted by r and the capital will thus be multiplied by r2
. Then we have
the relation:
r2
= r1r2 ⇔ r =
√
r1r2 (4.7.34)
This is an example where we therefore see the geometric mean. Forgetting to use
of the geometric mean a common mistake in companies where some employees
calculate the arithmetic average rate of increase of a reference value.
D7. The moving average of order n is defined as:
MMn =
x1 + x2 + ... + xn
n
(4.7.35)
The moving average is used particularly in economics, where it represents a trend of a
series of values, where the number of points is equal to the total number of points of the
serie of values less the number that you specify for the period.

Draft
A moving average in finance is calculated from the average of a stock price over a given
period: each point of a moving average of 100 sessions is the average of 100 last current
values. This curve, displayed simultaneously with the evolution of the curve of the values,
smooths the daily changes in the value and gives the possibility to better see the trends.
The moving averages can be calculated for different time periods, which can generate
short-term trends MMC (20 sessions according to the habits of the domain), medium
(50-100 sessions) or long-term MML (over 100 sessions):
Figure 4.66 – Graphical representation of a few moving averages
The crosses of the moving averages with the price curve (cutted with a certain granularity)
of the value generate purchase or sale (basic) signals depending on the case:
• Buy signal: when the price curve crosses the MM up.
• Sell signal: when the price curve crosses the MM down.
In addition to the moving average, note that there are a lot of other artificial indicators
often used in finance (the R software has a package dedicated only to such indicators).
As for example the upside/downside ratio.
The idea is the following: If you have a financial product (see section Economy) whose
current price is Pc for which you have a goal of high gain corresponding to a high price,
which we will denote by Ph (high price) and conversely, the potential loss that you feel is
at a price Pl (low price).
Ph − Pc
Pc − Pl
= UDR (4.7.36)

Draft
Examples:
E1. For example, a financial product of 10.− with a low price of 5.− and a
high price of 15.− has a ratio of UDR = 1 and therefore an identical speculative
factor to allow a gain or loss of 5.−.
E2. A financial product of 10.− with a low price of 5.− and a high price of 20.−
has a ratio of UDR = 2 and therefore twice the speculative potential gain compared
to the loss.
Remark
Some financial institutions recommend to refuse equation below 3. Investors also
tend to reject too high equation that can be a sign of artificial inflation.
D8. The weighted average (the moving average and arithmetic average are just a special
cases of the weighted average with wi = 1) is defined by:
µp =
n
i=1
wixi
n
i=1
wixi
(4.7.37)
Is used for example in geometry to locate the centroid of a polygon, in physics to de-
termine the center of gravity or in statistics to calculate the mean and other advanced
regression techniques and in project management for estimating task durations forecast.
In the general case the weights wi represents the weighted influence or arbitrary/empirical
influence of the element xi relatively to the others one.
D9. The functional mean or integral average is defined as:
µf =
1
b − a
bˆ
a
f(x)dx (4.7.38)
where µf depends of a function f of a real integrated variable (see Differential and Inte-
gral Calculus) on a range [a, b]. It is often used in signal theory (electronics, electrotech-
nichs).

Draft
4.7.2.1 Laplace Smoothing
To come back to our class frequencies seen above and before proceeding with the study of some
mathematical properties of the averages... you should know that when we work with discrete
probabilities distributions it happens very (very) often that we meet a typical problem whose
source is the size of the population.
Consider as an example the case where we have 12 documents and that we would like estimate
the probability of occurrence of the word Viagra. We have on a sample the following values:
Document ID Word occurences
1 1
2 0
3 2
4 0
5 4
6 6
7 3
8 0
9 6
10 2
11 0
12 1
Table 4.19 – Class frequencies of the word
Table that we can represent in another way:
Word occurences Documents Probability
0 4 0.33
1 2 0.17
2 2 0.17
3 1 0.083
4 1 0.083
5 0 0
6 2 0.17
Table 4.20 – Respective frequencies classes of documents
And here we have a common phenomenon. There is no record with 5 occurrences of the word
of interest. The idea (very common in the ﬁeld of Data Mining) is then to add artiﬁcially and
empirically using a count using a technique called Laplace smoothing which involves adding
k units at each occurrence. Therefore the table becomes:

Draft
Word occurences Documents Probability
0 5 0.26
1 3 0.16
2 3 0.16
3 2 0.11
4 2 0.11
5 1 0.05
6 3 0.16
Table 4.21 – Frequencies classes of documents with smoothing
Obviously this type of technique is debatable and beyond the scientific framework ... We even
hesitated to introduce this technique in the chapter of Numerical Methods (with the rest of all
the empirical numerical techniques) rather than here...
4.7.2.2 Means and Averages properties
Now we will see some relevant properties that connect some of these means and averages or are
specific to a particular mean/average.
The first properties are important so beware to understand them:
P1. The calculation of the arithmetic, root mean square and harmonic average/mean can be
generalized using the following expression:
µ = m
1
n
n
i=1
xm
i (4.7.39)
where we see:
(a) For m = 1, we get the arithmetic average
(b) For m = 2, we get the root mean square
(c) For m = −1 we get the harmonic mean
P2. The arithmetic average has the property of linearity, that is to say (without proof because
it is simple to check):
λx + µ = λ¯x + µ (4.7.40)
This is the statistical version of the property of the mean in the field of probabilities that
we will see further.
P3. The weighted sum of the deviations from the arithmetic average is zero.

Draft
Proof 4.22.2. First, by deﬁnition, we know that:
n =
r
i=1
ni and µ =
1
n
r
i=1
nixi (4.7.41)
then we have:
r
i=1
ni(xi−µ)=
r
i=1
nixi−µ
r
i=1
ni=
r
i=1
nixi−
1
n
r
i=1
nixi n=
r
i=1
nixi−
r
i=1
nixi=0
(4.7.42)
Thus, this tool can not be used as a measure of dispersion!
By extension, the arithmetic average of the weighted deviations from the average is also
equal to zero:
r
i=1
ni(xi − µ)
n
= 0 (4.7.43)
Q.E.D.
This result is quite important because it will further be useful for a better understanding
of the concept of standard deviation and variance.
P4. Now we would like to prove that:
µh µg µa µq (4.7.44)
Remark
The comparisons between the above means/averages and the median or the
weighted or moving averages does not make sense this is why we won’t compare
them.
Proof 4.22.3. First, we consider two nonzero real numbers x1 and x2 as x2 x1 0 and
then we write:
(a) The arithmetic average:
µa =
x1 + x2
2
(4.7.45)
(b) The geometric mean:
µg =
√
x1x2 (4.7.46)
(c) The harmonic mean:
1
µh
=
1
x1
+
1
x2
2
=
x2
x1x2
+
x1
x2x1
2
=
x1 + x2
2x1x2
⇔ µh =
2x1x2
x1 + x2
(4.7.47)

Draft
(d) The root mean square:
µ2
q =
x2
1 + x2
2
2
(4.7.48)
We will start to prove that µg µh by contradiction by putting µg − µh 0:
µg − µh =
√
x1x2 −
2x1x2
x1 + x2
=
√
x1x2x1 +
√
x1x2x2 − 2x1x2
x1 + x2
0
⇒
√
x1x2x1 +
√
x1x2x2 − 2x1x2 0
⇒
√
x1x2x1 +
√
x1x2x2 2x1x2
⇒
√
x1x2
x2
+
√
x1x2
x1
2
⇒
x1
x2
+
x2
x1
2
(4.7.49)
By convenience we will now put:
y =
x2
x1
(4.7.50)
and we know that y 1. We therefore have:
x1
x2
+
x2
x1
=
1
y
+ y (4.7.51)
and remember we search if it is possible that:
x1
x2
+
x2
x1
2 (4.7.52)
We can now easily check this statement from the following equivalences:
x1
x2
+
x2
x1
=
1
y
+ y =
y2
+ 1
y
2 ⇔ y2
+ 1 − 2y 0 ⇔ (y − 1)2
0 (4.7.53)
There is also a contradiction, and this validates our initial hypothesis:
µg − µh 0 ⇔ µg µh (4.7.54)
Let see if µg µa.
Under the hypothesis x2 x1 0. We search now to prove that:
x1 + x2
2

√
x1x2 (4.7.55)
Now we have the following equivalences:
x1 + x2
2

√
x1x2 ⇔ (x1 + x2)2
4x1x2 ⇔ x2
1 + x2
2 − 2x1x2 0 ⇔ (x1 − x2)2
0
(4.7.56)

Draft
and the last expression is obviously correct because the square of a (real) number is always
positive which verifies our initial hypothesis:
µa − µg 0 ⇔ µa µg (4.7.57)
We will prove now that µq mua by contradiction by putting µq − µa 0:
µq − µa =
x2
1 + x2
2
2
−
x1 + x2
2
⇒
x2
1 + x2
2
2
−
x1 + x2
2
0
⇒
x2
1 + x2
2
2

x1 + x2
2
⇒
x2
1 + x2
2
2

x2
1 + 2x1x2 + x2
2
4
⇒ x2
1 + x2
2
x2
1 + 2x1x2 + x2
2
2
⇒ x2
1 − 2x1x2 + x2
2 0
⇒ (x1 − x2)2
0
(4.7.58)
But the square of a (real) number is always positive which verifies our initial hypothesis:
µq − µa 0 ⇔ µq µa (4.7.59)
We then have:
µh µg µa µq (4.7.60)
Q.E.D.
Once these inequalities proved, we can then move on to a figure that we attribute to
Archimedes to place three of these averages. The interest of this example is to show that
there are some remarkable relations between statistics and geometry (coincidence??).
Figure 4.67 – Starting point for the geometric representation for the various averages

Draft
We will first write a = AB, b = BC and O is the midpoint of AC. Thus, the circle is
drawn with center O and radius OA. D is the intersection of the perpendicular segment
AC through B and of the circle Ω (we can choose the intersection we want). H is itself
the orthogonal projection of B on OD.
Archimedes says that OA is the arithmetic average of a and b and that BD is the geometric
mean of a and b, and DH is the harmonic mean of a and b.
We then prove that (could be trivial):
OA =
AC
2
=
a + b
2
(4.7.61)
Therefore OA is the arithmetic average µa of a and b.
We have in the right-angled triangle ADB:
AD
2
= DB
2
+ BA
2
(4.7.62)
Then we have in the right-angled triangle BDC:
DC
2
= BC
2
+ DB
2
(4.7.63)
We then add these two relations, and we get:
2DB
2
+ BA
2
+ BC
2
= AD
2
+ DC
2
(4.7.64)
We know that D is on a circle of diameter AC, so ADC is rectangle on D. Therefore:
AD
2
+ DC
2
= AC
2
(4.7.65)
And then we replace BA and BC by a and b:
2DB
2
+ a2
+ b2
= AC
2
= (a + b)2
⇔ 2DB
2
= 2ab ⇔ (4.7.66)
So finally:
DB =
√
ab (4.7.67)
And therefore, DB is the geometric mean µg of a and b. We have now prove that DH is
the harmonic mean of a and b. We have in a first time using the orthogonal projection as
study in the section of Vector Calculus:
−−→
DO ◦
−−→
DB = DO DB cos(α) = DO · DH =
a + b
2
DH (4.7.68)
Then we also have (also orthogonal projection)
−−→
DO ◦
−−→
DB = ( DO cos(α)) DB = DB · DB = DB
2
(4.7.69)
Therefore we have:
DB
2
=
a + b
2
DH (4.7.70)
and since DB =
√
ab, we have then:
DH =
2ab
a + b
(4.7.71)
DH is therefore the harmonic mean of a and b. Archimedes was not wrong!

Draft
4.7.3 Type of variables
In talking about variables quantitative or qualitative variables, sometimes you hear variables
being described as categorical (or sometimes nominal), or ordinal, or interval. Below we will
define these terms and explain why they are important.
Definitions:
D1. The discrete variables (by counting) that belongs to Z: Are analyzed with statistical
laws based on a countable definition domain always strictly positive (the Poisson or Hy-
pergeoemetric distribution are such typical case in the industry). Are almost always rep-
resented graphically by histograms.
D2. The continuous variables (by measure) that belong to R: Are analyzed with statisti-
cal laws based on an uncountable domain of definition strictly positive or may take any
positive or negative value (typically the Normal distribution in the industry). Are almost
always represented graphically by histograms with class intervals.
D3. The attribute variables (by classification): They are not digital data (only when they
are coded with digits!) but qualitative data type Yes, No, Passed, Failed, On time, Late,
red, green blue, black, etc. The binary data type attribute follow a Bernoulli while higher
order qualitative variables have no average or standard deviation (effectively... try to
calculate the mean and standard deviation between the qualitative variables Red, Green
and Pink...).
In attribute variable we mainly distinct two subtypes of variables:
(a) A categorical variable (sometimes called a nominal variable) is one that has two
or more categories, but there is no intrinsic ordering to the categories. For example,
gender is a categorical variable having two categories (male and female) and there
is no intrinsic ordering to the categories. Hair color is also a categorical variable
having a number of categories (blonde, brown, brunette, red, etc.) and again, there
is no agreed way to order these from highest to lowest. A purely categorical variable
is one that simply allows you to assign categories but you cannot clearly order the
variables. If the variable has a clear ordering, then that variable would be an ordinal
variable, as described below.
(b) An ordinal variable is similar to a categorical variable. The difference between the
two is that there is a clear ordering of the variables. For example, suppose you
have a variable, economic status, with three categories (low, medium and high). In
addition to being able to classify people into these three categories, you can order
the categories as low, medium and high. Now consider a variable like educational
experience (with values such as elementary school graduate, high school graduate,
some college and college graduate). These also can be ordered as elementary school,
high school, some college, and college graduate.
Understanding the different types of data is an important discipline for the engineers because it
has important implications for the type of analysis tools and techniques that will be used.

Draft
A common question regarding the collection of data is what is the amount that should be col-
lected. In fact it depends on the desired level of accuracy. We will see much further in this
section (with proof!) how to mathematically determine the amount of data to collect.
Now that we are relatively familiar with the concept of average (mean), we can discuss on more
formal calculations and that will make sense.
4.7.3.1 Discrete Variables and Moments
Consider X is an independent variable (an individual of a sample, whose property is inde-
pendent of other individuals) that can take discrete random values (realizations of the vector
(X1, X2, ..., Xn)) with respective probabilities (p1, p2, ..., pn) where, by the axioms of probabil-
ities (see section Probabilities):
pi ∈ [0, 1]
i
pi = 1 (4.7.72)
Definitions:
D1. Let X be a numeric (quantitative) random variable (r.v.). It will be fully described in prac-
tice most of time by the value of the probability (for discrete variables) for a realization
of this variable or by the cumulative probability (for discrete AND continuous variables)
to be typically less than or equal X for all realizations x. This cumulative (cumulative) is
denoted by:
F(x) = P(x ≤ X)ı∀x ∈ R (4.7.73)
with:
1 ≥ P(X) ≥ 0 and 1 ≥ F(X) ≥ 0 (4.7.74)
where F(x) is named the repartition fonction of the random variable X. It is the theo-
retical proportion of the population whose value is less than or equal to x. It follows for
example:
P(X x) = 1 − F(x) ⇔ P(X ≤ x) + P(X x) = 1 (4.7.75)
More generally, for any two numbers a and b with a b, we have:
P(a ≤ x ≤ b) = F(b) − F(a) (4.7.76)
D2. The empirical repartition function is naturally defined by (we have indicated the differ-
ent notations that you can found in the literature):
ˆF(x) =
1
n
n
i=1
1xi≤x =
1
n
n
i=1
δxi≤x =
#(i : xi ≤ x)
n
(4.7.77)
associated with the sample of independent and identically distributed variables which as
we know is named a random vector denoted by (x1, x2, ..., xn). It is simply the cumula-
tive frequencies of appearance normalized to unity below a certain fixed value (approach

Draft
that the majority of human beings are naturally using when seeking the repartition func-
tion).
So if we take again the example of wages already used above, then we have for example
for x fixed to 1,800:
Ordered Wages xi ≤ x Frequencies 1xi≤x
1,200 1
1,220 1
1,250 1
1,300 1
1,350 1
1,450 1
1,450 1
1,560 1
1,600 1
1,800 1
1,900 0
2,150 0
2,310 0
2,600 0
3,000 0
3,400 0
4,800 0
Table 4.22 – Example of the empirical repartition function
And then:
ˆF17(x ≤ 1, 800) =
1
17
17
i=1
1xi≤x =
1
17
10 ∼= 59% (4.7.78)
The repartition function is clearly a monotonically increasing function (or more precisely
non-decreasing) whose values range from 0 to 1.
4.7.3.1.1 Mean and Deviation of Discrete Random Variables
Definition: We define the expectation or mean, also named moment of order 1, of
the random variable X by the relation (with various notations):
µX = E(X) = E(X) =
i
pixi
(4.7.79)
also sometimes named parts rule.
In other words, we know that for every event in the sample space is associated with a probability
that we also associate with a value (given by the random variable). The question then is know
what value we can get at long term? The expected value (the mean...) is then the weighted
average, by the probability, of all values of the events of sample space.

Draft
If the probability is given by a discrete distribution function f(xi) (see the deﬁnitions of distri-
bution functions later below in the text) of the random variable, we then have:
pi = f(xi) ⇒ µX = E(X) =
i
xif(xi) (4.7.80)
Remark
R1. The mean µX can also be written just simply µ if there are no possible confusion on
the random variable.
R2. If we consider each realization of the random variables (x1, x2, ..., xn) as the
components of a vector x and each associated probability (or ponderation) (p1, p2, ..., pn)
as the components of a vector p we can write the mean in a technical way using the
scalar product (Vector Calculus) often written:
p ◦ x = p, x =
n
i=1
pixi = E(X) (4.7.81)
Here are the most important mathematical properties of the mean for any random variable
(whatever the distribution law!) and that we will often use throughout this section (and many
other involving statistics):
P1. Multiplication by a constant (homogeneous):
E(αX) =
i
αxiP(X = xi) = α
i
xiP(X = xi) = αE(αX) (4.7.82)
P2. Sum of two random variables (independant or not!):
E(X + Y ) =
i,j
[(xi + yj)P ((X = xi) ∩ (Y = yi))]
=
i,j
[xiP ((X = xi) ∩ (Y = yj))] +
i,j
[yiP ((X = xi) ∩ (Y = yj))]
=
i
xi
j
P ((X = xi) ∩ (Y = yi)) +
j
yj
i
P ((X = xi) ∩ (Y = yj))
=
i
xiP (X = xi) ∩
j
(Y = yi) +
j
yjP (Y = yj) ∩
i
(X = xi)
=
i
xiP(X = xi) +
j
yjP(Y = yj) = E(X) + E(Y )
(4.7.83)
Where we used in the 4th line, the property view in the section of Probabilities:
P
i∈N
Ai =
i∈N
P(Ai) (4.7.84)

Draft
We deduce that for n random variables Xi following any probability distribution:
E (Xi) =
i
E(Xi) (4.7.85)
P3. Then mean of a constant a is equal to the constant itself:
E(a) =
i
api = a
i
pi = a · 1 = a (4.7.86)
P4. Mean of a product of two random variables:
E(X · Y ) =
i,j
xiyiP(X = xi, Y = yj) (4.7.87)
And if the two random variables are independent, then the probability is equal to the joint
probability (see section Probabilities). Therefore we have:
E(X · Y ) =
i,j
xiyjP(X = xi, Y = yj) =
i,j
xiyjP(xi)P(yj) =
i,j
xixiP(xi)yiP(yi)
=
i,j
xiP(xi)
i,j
yiP(yi) = E(X)E(Y )
(4.7.88)
So the mean of the product of independent random variables is always equal to the product of
their means.
We will assume as obvious that these four properties extend to the continuous case!
Definition: After having translated the trend by the mean it is interesting to have and indicator
that reflects the dispersion or standard deviation around the mean by a value named variance
of X or second-order centered moment or mean square error (MSE), written V (X) or σ2
X
(read sigma square) and given in its discrete form by:
σ2
X = V(X) = MSE = E (X − µX)2
=
i
(xi − µX)2
f(xi) =
i
(xi − µX)2
pi (4.7.89)
The variance is however not directly comparable to the mean because of the fact that the units
of the variance are the square of the unit of the random variable, which follows directly from its
definition. To have an indicator of dispersion that can be compared to the parameters of central
tendency (mean, median and ... mode), it then suffices to take the square root of the variance.
For convenience, we define the standard deviation of X by:
σx = σ(X) = V(X) (4.7.90)
Remark
R1. The standard deviation σX of the random variable X can be written simple σ if there
is no possible confusion.
R2. The standard deviation and variance are, in the literature, often named dis-
persion parameters as opposed to the mean, mode and median that are named
positional parameters.

Draft
Definition: The ratio (expressed in %):
CV =
σ
µ (4.7.91)
is often used in business to compare the mean and the standard deviation and is named the co-
efficient of variation C.V. because it has no units (which is it’s main advantage!) and because
many industrial statistical methods consider that a good C.V should ideally be just about a few
% only.
Thus, in practice we consider that:
Coefficient of variation Quality
20% Poor
10% Acceptable
5% Controlled
2.5% Excellent
1.25% World Class
0.0625% Rarely achieved
Table 4.23 – Qualitative judgments of C.Vs commonly accepted
Why do we find a square (respectively a square root) in the definition of the variance? The
intuitive reason is simple (the rigorous much less ...). Remember, that we have shown above
that the sum of the deviations from the actual weighted average is always zero:
r
i=1
ni(xi − µ) = 0 (4.7.92)
If we assimilate the size of each sample by the probability by normalizing the sample size with
respect to n, we come upon a relation that is the same as the variance with the difference that
the term in brackets is not squared. And then we immediately see the problem... the dispersion
measure is always zero, hence the need to bring this to the square.
We could, however, imagine to use the absolute value of deviations from the mean, but for a
number of reasons that we will see later during our study of estimators, the choice of squaring
is quite natural.
Note, however, the common use in the industry of two common other indicators of dispersion:
1. The mean absolute deviation (mean of the absolute values of deviations from the mean):
AVEDEV =
n
i=1
| xi − µ |
n
(4.7.93)
Which is a very elementary indicator used when we do not want to make statistical in-
ference on a series of measures. This deviation can be easily calculated in the English
version of Microsoft Excel 11.8346 using the e†ihi†@ A function.

Draft
2. The median absolute deviation noted MAD (median of absolute values of deviations
from the median):
MAD = Me (| X − Me(X) |) (4.7.94)
which is considered as a more robust measure of dispersion than those given by the mean
absolute deviation or the standard deviation (unfortunately this indicator is not natively
integrated in spreadsheets softwares).
Example:
Consider the following measure of a random variable X:
(1, 1, 2, 2, 4, 6, 9) (4.7.95)
and where the median value is given as we know by:
Me(X) = Me(1, 1, 2, 2, 4, 6, 9) = 2 (4.7.96)
The absolute deviations from the median are then:
| X − Me(x) |= (1, 1, 0, 0, 2, 4, 7) (4.7.97)
Placed in ascending order, we then have:
(0, 0, 1, 1, 2, 4, 7) (4.7.98)
where we easily identify the absolute deviation from the median, which is:
MAD = Me(0, 0, 1, 1, 2, 4, 7) = 1 (4.7.99)
In the case where we have at disposition a series of measures, we can estimate the experimental
value of the mean (expectation) and of the variance with the following estimators (it is simply
the of average and standard deviation of a sample when the events are equally likely) with the
speciﬁc notation:
ˆµ =
n
i=1
xi and ˆσ2
=
1
n
n
i=1
(xi − ˆµ)2
(4.7.100)
Proof 4.22.4. First for the mean:
µ = E(X) =
n
i=1
pixi =
n
i=1
1
ni
xi =
1
n
n
i=1
xi = ˆµ (4.7.101)
And for the variance:
σ2
= E((X − E(X))2
) = E((X − ˆµ)2
) =
n
i=1
pi(xi − µ)2
=
n
i=1
1
ni
(xi − µ)2
=
1
n
n
i=1
(xi − ˆµ)2
= ˆσ2
(4.7.102)

Draft
Q.E.D.
Theorem 4.23. Let us prove now a very nice little property as the arithmetic average is an
optimum for the sum of squared errors.
Proof 4.23.1.
n
i=1
(xi − α)2
=
n
i=1
x2
i − 2α
n
i=1
xi + nα2
(4.7.103)
And if we search for α as the derivative of the above expression is equal to zero:
d
dα
n
i=1
x2
i − 2α
n
i=1
xi + nα2
= 0 (4.7.104)
then α is an optimum. We have therefore:
d
dα
n
i=1
x2
i − 2α
n
i=1
xi + nα2
= −2
n
i=1
xi + 2nα = 0 (4.7.105)
or after rearrangement and an elementary simplification we get:
α =
1
n
n
i=1
xi (4.7.106)
Q.E.D.
It is effectively the arithmetic average! Now to see if it is an maximum extrema or minimum
extrema we just need calculate the second derivative (see section Differential and Integral Cal-
culus) and see if it gives a positive constant (i.e. the first derivative increases when α increase).
Therefore we immediately see that it is effectively a minimum extrema!!!
Remark
The term of the sum that we see in the expression of the variance (standard deviation) is
named the sum of squared deviations from the mean or sum of squared errors from
the mean. We also name it the total sum of squares, or total variation, or sum of
square errors in the context of the study of the ANOVA (see the further below)
Before that we continue, let us recall the concept of geometric mean seen above (widely used
for returns in finance or growth analyzes in % of sales):
µg = n
n
i=1
xi =
n
i=1
xi
1
n
(4.7.107)

Draft
It’s fine but employees in financial departments also need to calculate the standard deviation of
this average. The idea is then to take the logarithm to reduce it to a simple arithmetic mean (it
is still obviously an estimator!):
ln(µg) = ln




n
i=1
xi
1
n



 =
1
n
ln
n
i=1
xi =
1
n
n
i=1
ln(xi) (4.7.108)
Therefore, since taking the logarithm of the values we have the arithmetic mean of the log
values, then the logarithm of the geometric standard deviation (with physicist reasoning like...)
will be:
ln (ˆσg) =
1
n
n
i=1
(ln(xi) − ln(µg)) =
1
n
n
i=1
ln
xi
µg
(4.7.109)
Then we just take the exponential of the standard deviation of the logarithms of the values to
have the geometric standard deviation:
ˆσg = e
1
n
n
i=1
ln
xi
µg
(4.7.110)
The variance can also be written in the very important way named the Huyghens relation or
König-Huyghens theorem or Steiner translation theorem that we will reuse several times
thereafter. Let’s see what it is:
V(X) = E[(X − µX)2
] =
i
(xi − µX)2
f(xi) =
i
x2
i − 2µXxi + µ2
X f(xi)
=
i
x2
i f(xi) − 2µX
i
xif(xi) + µ2
X
i
f(xi) =
i
x2
i f(xi) − 2µ2
X + µ2
X
=
i
x2
i f(xi) − µ2
X = E(X2
) − µ2
X = E(X2
) − E(X)2
(4.7.111)
Let us now do a relatively small hook to a common scenario generator of errors in business when
several statistical series are handled (very common case in the industry as well as in insurance
or finance)!
Consider two data series on the same character:
• (x1, n1), (x2, n2), ..., (xp, np) sample of total size n, arithmetic average ¯x, standard devi-
ation σx.
• (y1, m1), (y2, m2), ..., (yp, mq) sample of total size m, arithmetic average ¯y, standard de-
viation σy.
We then have:
¯z =
p
i=1
xi +
q
i=1
yi
n + m
=
n
1
n
p
i=1
xi + m
1
m
q
i=1
yi
n + m
=
n¯x + m¯y
n + m
(4.7.112)

Draft
So the average of the averages is not equal to the overall average (ﬁrst common mistake in
business) except if the two data series have the same sample size (n = m)!!!
Let have a look at the standard deviation always with the same situation. First remember that
we have:
σ2
x =
p
i=1
pi(xi − ¯x)2
=
1
n
p
i=1
ni(xi − ¯x)2
=
1
p
i=1 ni
p
i=1
ni(xi − ¯x)2
σ2
y =
q
i=1
pi(yi − ¯y)2
=
1
m
q
i=1
mi(yi − ¯y)2
=
1
q
i=1 mi
q
i=1
mi(yi − ¯y)2
(4.7.113)
To continue, recall that we have previously proved the Huyghens theorem and therefore:
V(Z) = σz = E(Z) − E(Z)2
=
1
k
k
i=1
kiz2
i − ¯z2
(4.7.114)
Therefore we have:
σ2
z =
1
k
k
i=1
kiz2
i − ¯z2
=
1
k
k
i=1
kiz2
i −
n¯x + m¯y
n + m
2
=
1
n + m
p
i=1
nix2
i +
q
i=1
miy2
i −
n¯x + m¯y
n + m
2
=
n
1
n
p
i=1
nix2
i + m
1
m
q
i=1
miy2
i
n + m
−
n¯x + m¯y
n + m
2
=
n
1
n
p
i=1
nix2
i − n¯x2
+ m
1
m
q
i=1
miy2
i − m¯y2
n + m
+
n¯x + m¯y2
n + m
−
n¯x + m¯y
n + m
2
=
n
1
n
p
i=1
nix2
i − ¯x2
+ m
1
m
q
i=1
miy2
i − ¯y2
n + m
+
n¯x + m¯y2
n + m
−
n¯x + m¯y
n + m
2
=
nσ2
x + mσ2
y
n + m
+
n¯x + m¯y2
n + m
−
n¯x + m¯y
n + m
2
=
nσ2
x + mσ2
y
n + m
+
(n¯x2
+ m¯y2
)(n + m) − (n¯x + m¯y2
)
(n + m)2
=
nσ2
x + mσ2
y
n + m
+
(n2
¯x2
+ mn¯y2
+ mn¯x2
+ m2
¯y2
) − (n2
¯x2
+ 2n¯xm¯y + m2
¯y2
)
(n + m)2
(4.7.115)
And we continue on the next page...:

Draft
σ2
z =
nσ2
x + mσ2
y
n + m
+
mn¯y2
+ mn¯x2
− 2n¯xm¯y
(n + m)2
=
nσ2
x + mσ2
y
n + m
+ nm
¯y2
+ ¯x2
− 2¯x¯y
(n + m)2
=
nσ2
x + mσ2
y
n + m
+ nm
(¯y + ¯x)2
(n + m)2
(4.7.116)
So we see that the overall standard deviation is not equal to the sum of the deviations (second
common mistake in business) unless the sample sizes and arithmetic averages are the same in
both series (that is to say n = m and ¯x = ¯y)!!!
Consider now X being a random variable of mean µ (constant and determined value) and vari-
ance σ2
(constant and determined value), we define the reduced centered variable by the
relation:
Y =
X − µ
σ
(4.7.117)
Theorem 4.24. We prove in a very simple way by using the property of linearity of the mean
and property of scalar multiplication of the variance that:
E(Y ) = 0, V(Y ) = 1 (4.7.118)
Proof 4.24.1. For the proof we just use the definitions of the expected mean and variance (using
Huygens theorem for this latter). So let us begin with the mean:
E(Y ) = E
X − µ
σ
=
1
σ
E (X − µ) =
1
σ
(E(X) − E(µ)) =
1
σ
(µ − µ) = 0 = µY
(4.7.119)
And now with the variance using the Huyghens theorem:
V(Y ) = E(Y 2
) − E(Y )2
= E(Y 2
) − µ2
Y = E(Y 2
) − 02
= E(Y 2
)
= E
X − µ
σ
2
= E
1
σ2
X2
− 2Xµ + µ2
=
1
σ2
E(X2
) − 2µE(X) + µ2
=
1
σ2
E(X2
) − 2µµ + µ2
=
1
σ2
E(X2
) − µ2
=
1
σ2
V(X) + E(X)2
− µ2
=
1
σ2
V(X) + µ2
− µ2
=
1
σ2
V(X) =
1
σ2
σ2
= 1
(4.7.120)
Q.E.D.
Thus, any statistical distribution defined by a mean and standard deviation can be transformed
into another distribution often easier to analyze statistical. Therefore making this transforma-
tion, we obtain a random variable for which the parameters of the distribution low are now
useless to know. When we do that with other laws, and in the general case, when we speak of
pivotal variables.
Here are some important mathematical properties of the variance:

Draft
P1. Multiplication by a constant:
V(αX) =
i
f(xi)(αxi − αµ)2
= α2
i
f(xi)(xi − µ)2
= α2
V (4.7.121)
P2. Sum of two random variables:
V (X + Y ) =
i j
((xi + yi) − (µX + µY ))2
f(xi, yi)
=
i j
((xi − µX) + (yi − µY ))2
f(xi, yi)
=
i j
(xi − µX)2
+ 2(xi − µX)(yi − µY ) + (yi − µY )2 2
f(xi, yi)
=
i,j
(xi − µX)2
f(xi, yi) + 2
i,j
(xi − µX)(yi − µY )f(xi, yi)
+
i,j
(yi − µY )2
f(xi, yi)
= V(X) + 2
i,j
(xi − µX)(yi − µY )f(xi, yi) + V(Y )
= V(X) + 2E [(X − µX)(Y − µY )] + V(Y )
:= V(X) + 2cov(X, Y ) + V(Y )
(4.7.122)
Where we meet for the ﬁrst time the concept of covariance denoted by cov().
P4. Product of two random variables (using the Huyghens theorem):
V(X · Y ) = E (XY )2
− E(XY )2
= E(X2
Y 2
) − E(XY )2
(4.7.123)
And if the two random variables are independent, we get:
V(X · Y ) = E X2
E Y 2
− E (X)2
E (Y )2
(4.7.124)
What we can rewrite using once again the Huyghens theorem:
V(X · Y ) = (V(X) − E(X)2
)(V(Y ) − E(Y )2
) − E (X)2
E (Y )2
(4.7.125)
We will assume as obvious that these four properties extend to the continuous case!

Draft
4.7.3.1.2 Discrete Covariance
We have seen in on of the last equations the concept of covariance for which we will
determine a more convenient expression later:
cov(X, Y ) = cX,Y = 2E [(Xi − µX)(Y − µY )] (4.7.126)
We introduce now a more general and very important expression of the covariance in many
application ﬁelds:
V(X + Y + Z) =
i j k
((xi + yj + zk) − (µX + µY + µZ))2
f(xi, yj, zk)
=
i j k
((xi − µX) + (yj − µY ) + (zk − µZ))2
f(xi, yj, zk)
=
i j k
(A + B + C)2
f(xi, yj, zk)
=
i j k
(A2
+ B2
+ C2
+ 2AB + 2BC + 2AC)f(xi, yj, zk)
= V(X) + V(Y ) + V(Z) + 2
i j k
(AB + BC + AC)f(xi, yj, zk)
= V(X) + V(Y ) + V(Z) + 2cov(X, Y ) + 2cov(Y, Z) + 2cov(X, Z)
(4.7.127)
Now we change the notation to simplify even more:
V(X1 + X2 + X3) = V(X1) + V(X2) + V(X3) + 2cov(X1, X2) + 2cov(X2, X3) + 2cov(X1, X3)
=
3
i=1
V(Xi) + 2
3
ij
cov(Xi, Xj)
(4.7.128)
Therefore in the general case:
V
n
i=1
Xi =
n
i=1
V(Xi) + 2
n
ij
cov(Xi, Xj) (4.7.129)
Or using standard deviation:
σ n
i=1 Xi
=
n
i=1
σ2
Xi
+ 2
n
ij
cov(Xi, Xj) (4.7.130)
Using the properties of the mean (especially E(X) = cte
and E(cte
) = cte
) we can write the
covariance in a much simpler way for calculation purposes:
cov(X, Y ) = E [(X − E(X))(Y − E(Y ))] = E [XY − E(X)Y − XE(Y ) + E(X)E(Y )]
= E(XY ) − E(E(X)Y ) − E(XE(Y )) + E(E(X)E(Y ))
= E(XY ) − E(X)E(Y ) − E(X)E(Y ) + E(X)E(Y )
= E(XY ) − E(X)E(Y )
(4.7.131)

Draft
and we obtain the relation widely used in statistics and ﬁnance in the practice called the co-
variance formula...:
cX,Y = cov(X, Y ) = E(XY ) − E(X)E(Y ) (4.7.132)
which is however best known when written as:
cX,Y = cov(X, Y ) =
1
n i
xiyi − ¯x¯y (4.7.133)
If X = Y (equivalent to a univariate covariance) we fall back again on the Huyghens theorem:
cX,X = cov(X, X) = E(XX) − E(X)E(X) = E(X2
) − E(X)2
(4.7.134)
Remark
Statistics can be partitioned according to the number of random variables we study. Thus,
when a single random variable is studied, we speak of univariate statistics, for two
random variables of bivariate statistics and in general, of multivariate statistics.
If and only if the variables are equally likely, we ﬁnd the covariance in the literature in the
following form, sometimes named Pearson covariance, which derives from calculations that
we have done previously with the mean:
cX,Y =
1
n
n
i=1
(yi − µY )(xi − µX) (4.7.135)
Covariance is a measure of the simultaneous variation of X and Y . Indeed, if X and Y generally
grow simultaneously, the products (yi − µY )(xi − µX) will be positive (positively correlated),
whereas if Y decreases as X increases, these same products will be negative (negative correla-
tion). Note that if we distribute the terms of the last equation, we have:
cX,Y = cov(X, Y ) =
1
n
n
i=1
(yi − µY )(xi − µX) =
1
n
n
i=1
(yi − ¯y)(xi − ¯x)
=
1
n
n
i=1
xi(yi − ¯y) − ¯x(yi − ¯y) =
1
n
n
i=1
xi(yi − ¯y) − ¯x
n
i=1
(yi − ¯Y )
(4.7.136)
and we have already shown that the sum of the deviations from the mean is zero. Hence we get
another common way to write the covariance:
cX,Y = cov(X, Y ) =
1
n
n
i=1
xi(yi − ¯y) (4.7.137)
and by symmetry:
cX,Y = cov(X, Y ) =
1
n
n
i=1
yi(xi − ¯x) (4.7.138)

Draft
So in the end, in the equiprobable case, we ﬁnally have the equivalent three important relations
used in various sections of this book:
cX,Y = cov(X, Y ) =
1
n
n
i=1
(yi − ¯y)(xi − ¯x)
cX,Y = cov(X, Y ) =
1
n
n
i=1
xiyi − ¯x¯y
cX,Y = cov(X, Y ) =
1
n
n
i=1
xi(yi − ¯y) =
1
n
n
i=1
yi(xi − ¯x)
(4.7.139)
In the section Numerical Methods for the study of linear regression and factor analysis we will
need the explicit expression of the bilinearity property of the variance. To see what it is exactly,
consider three random variables X, Y and Z and two constants a and b. Then using the third
relation given above, we have:
cov (Y, aX + bZ) =
1
n i
(yi − ¯y)(axi + bzi) =
1
n i
[(yi − ¯y)axi + (yi − ¯y)bzi]
=
1
n i
(yi − ¯y)axi +
i
(yi − ¯y)bzi
= a
1
n i
(yi − ¯y)xi + b
1
n i
(yi − ¯y)zi
= acov(X, Y ) + bcov(Y, Z)
(4.7.140)
The last relation is also important and will be used in several sections of this book (Economy,
Numerical Methods). It also allows us to directly obtain the covariance for the sums of various
random variables.

Draft
Example:
If X, Y, Z, T are four random variables defined on the same population, we want
to compute the following covariance:
cov(3X + 5Y, 4Z − 2T) (4.7.141)
We will develop that in two phases (this is also why we call that bilinearity property).
First with respect to the second argument (random choice!):
cov(3X + 5Y, 4Z − 2T) = 4cov(3X + 5Y, Z) − 2cov(3X + 5Y, T) (4.7.142)
And then with respect to the first:
cov(3X + 5Y, 4Z − 2T) = 4 [3cov(X, Z) + 5cov(Y, Z)] − 2 [3cov(X, T) + 5cov(Y, T)]
(4.7.143)
So in the end:
cov(3X + 5Y, 4Z − 2T) = 12cov(X, Z) + 20cov(Y, Z) − 6cov(X, T) − 10cov(Y, T)
(4.7.144)
Now, consider a set of random vectors Xi := Xi of components (x1, x2, ..., xn)i. The calcula-
tion of the covariance of the components by pairs gives what is called the covariance matrix
(a tool widely used in finance, management and statistical numerical methods!).
Indeed, we define the component (m, n) of the covariance matrix by:
covXm,Xn = E [(Xm − µXm )(Xn − µXn )] := cm,n (4.7.145)
We can therefore write a symmetric matrix (usually in practice it must be a square matrix...) in
the form:
Σ =





c11 c12 · · · c1n
c21 c22 · · · c2n
...
...
...
...
cn1 cn2 · · · cnn





(4.7.146)
where Σ is the usual tradition letter to denote the covariance matrix. By symmetry and because
it is a square n by n matrix only the number
n(n + 1)
2
of components is useful for us to deter-
mine the whole matrix (trivial but important information for when we will study the structural
equation modeling in the Numerical Methods section).
This matrix has the remarkable property that if we take the set of all random vectors and we
calculate the covariance matrix, then the diagonal will give us obviously the variances of each
pair of vectors (see examples in the chapters Economics, Numerical Methods or Industrial En-
gineering) because we have for recall:
covXm,Xm = E [(Xm − µXm )(Xn − µXm )] = E (Xm − µXm )2
= V(Xm) = σ2
Xm
(4.7.147)

Draft
This is why this matrix is often named variance-covariance matrices and finds itself sometimes
also written as follows:
Σ =





V11 c12 · · · c1n
c21 V22 · · · c2n
...
...
...
...
cn1 cn2 · · · Vnn





=





σ2
11 c12 · · · c1n
c21 σ2
22 · · · c2n
...
...
...
...
cn1 cn2 · · · σ2
nn





(4.7.148)
And this is a little bit abusively sometimes written as:
Σ =





σ2
11 σ12 · · · σ1n
σ21 σ2
22 · · · σ2n
...
...
...
...
σn1 σn2 · · · σ2
nn





(4.7.149)
This matrix has the advantage of quickly showing what pairs of random variables have a nega-
tive covariance and there... for which random variable the variance of the sum is smaller than
the sum of the variances!
Remark
As we already mention it, this matrix is very important and we will often see it again
in the section Economy during our study of modern portfolio theory and also for data
mining techniques in the section of Numerical Methods (principal compoments analysis
for example but not only!) and also in Industrial Engineering during our study of bivariate
control charts.
Recall now that we have an axiom in probability (see section Probabilities) which stated that
two events A and B are independent if and only if:
P(A ∩ B) = P(A)P(B) (4.7.150)
Similarly, by extension, we define the independence of discrete random variables.
Definition: Let X, Y be two discrete random variables. We say that X, Y are independent if
and only if:
∀x, y ∈ R P(X = x, Y = y) = P(X = x)P(Y = y) (4.7.151)
More generally, the discrete variables X1, X2, ..., Xn are independent (in block) if:
∀x1, ..., xn ∈ R P(X1 = x1, X2 = x2, ..., X3 = x3) =
n
i=1
P(Xi = xi) (4.7.152)
Theorem 4.25. The independence of two random variables implies that their covariance is zero
(the opposite is false!).
Proof 4.25.1. We will prove this in the case where the random variables take only a finite
number of values {xi}I and {yj}J , respectively, with I, J finite sets.

Draft
For the proof let us recall that:
E(XY ) =
i,j
P(X = xi, Y = yj)xiyj =
i,j
P(X = xi)P(Y = yj)xiyj
=
i
P(X = xi)xi
j
P(Y = yj)yj = E(X)E(Y )
(4.7.153)
and therefore:
cX,Y = E(XY ) − E(X)E(Y ) = E(X)E(Y ) − E(X)E(Y ) = 0 (4.7.154)
Remark
So small is the covariance (near to zero), more the series are independent. Conversely,
the greater the covariance (in absolute value) higher the series are dependant.
Given that:
V(X + Y ) = V(X) + V(Y ) + 2cX,Y (4.7.155)
and the fact that if X and Y are independent we have cX,Y = 0. Then:
V(X + Y ) = V(X) + V(Y ) (4.7.156)
More generally if X1, ..., Xn are independent (in block) then for any discrete or continuous
statistical distribution law (!) we have using the two most common notations:
V
n
i=1
Xi =
n
i=1
V(Xi) (4.7.157)
Or using the standard deviation:
σ n
i=1 Xi
=
n
i=1
σ2
Xi
(4.7.158)
Q.E.D.

Draft
4.7.3.1.2.1 Anscombe’s famous quartet
Anscombe’s quartet comprises four datasets that have nearly identical elementary statis-
tical properties, yet appear very different when graphed or analyzed with undergraduate
statistics rather than high-school one. Each dataset consists of eleven (x, y) points. They were
constructed in 1973 by the statistician Francis Anscombe to demonstrate both the importance
of graphing data before analyzing it and the effect of outliers on statistical properties.
The datasets are as follows. The x values are the same for the ﬁrst three datasets:
I II III IV
x y x y x y x y
10.0 8.04 10.0 9.14 10.0 7.46 8.0 6.58
8.0 6.95 8.0 8.14 8.0 6.77 8.0 5.76
13.0 7.58 13.0 8.74 13.0 12.74 8.0 7.71
9.0 8.81 9.0 8.77 9.0 7.11 8.0 8.84
11.0 8.33 11.0 9.26 11.0 7.81 8.0 8.47
14.0 9.96 14.0 8.10 14.0 8.84 8.0 7.04
6.0 7.24 6.0 6.13 6.0 6.08 8.0 5.25
4.0 4.26 4.0 3.10 4.0 5.39 19.0 12.50
12.0 10.84 12.0 9.13 12.0 8.15 8.0 5.56
7.0 4.82 7.0 7.26 7.0 6.42 8.0 7.91
5.0 5.68 5.0 4.74 5.0 5.73 8.0 6.89
Table 4.24 – Anscombe’s quartet
The quartet is still often used to illustrate the importance of looking at a set of data graphically
before starting to analyze according to a particular type of relationship, and the inadequacy of
basic statistic properties for describing realistic datasets.
With Microsoft Excel 14.0.7166 we get:

Draft
Figure 4.68 – Anscombe’s quartet Statistics Summary
As we can see with elementary statistical indicators it is almost impossible to guess a difference
between the four data sets. But if we use the skewness or the kurtosis this change everything!
Looking to the corresponding charts we get the same conclusion:
Figure 4.69 – Anscombe’s quartet Graphs Summary

Draft
4.7.3.1.3 Mean and Variance of the Average
Often in statistics, it is (verrrrry!) useful to determine the standard deviation of the
sample mean and to work with it to get important analytical results in management and
manufacturing. Let’s see what it is!
Given the average of a series of terms, each determined by the measurement of several values
(it is in fact its estimator in a particular case as we will see later):
¯X =
1
n
(X1 + X2 + ... + Xn) (4.7.159)
then using the properties of the mean:
E ¯X =
1
n
(E(X1) + E(X2) + ... + E(Xn)) (4.7.160)
and if all the random variables are independent and identically distributed then we have:
E ¯X =
1
n
(µ + µ + ... + µ) =
1
n
nµ = µ (4.7.161)
Remark
We will prove much further below that if all the random variables are independent and
identically distributed with ﬁnite variance, then the mean follows asymptotically what
we name a Normal distribution.
For the variance, the same reasoning applies:
V( ¯X) = σ2
¯X =
1
n2
(V(X1) + V(X2) + ... + V(Xn)) =
1
n2
σ2
1 + σ2
2 + ... + σn
2 (4.7.162)
And if the random variables are independent and identically distributed (we will study further
the very important case current in practice where the last condition is not satisﬁed):
V( ¯X) = σ2
¯X =
σ2
n
(4.7.163)
Then we get the standard deviation of the mean also called standard error or non-systematic
variation:
σ ¯X =
σ2
√
n
(4.7.164)
and this is strictly the standard deviation of the estimator of mean!
This relation is available in many softwares including Microsoft Excel charts (but there is no
built-in function in Excel) and is written with the standard deviation (as above) or with the
notation of the variance (then we only have to take the square root...).
Note that the last relation can be used even if the average of n random variables is not the same!
The main condition is just that the standard deviations are all equal and this is the case in the
industry (production).

Draft
We then have:
E(Sn) = nµ E(Mn) = µ
V(Sn) = σ2
Sn
= nσ2
V(Mn) = σ2
Mn
=
σ2
n
(4.7.165)
where Sn is the sum of n independent identically distributed random variables and Mn their
estimated average.
The reduced centered variable that we introduced earlier:
Y =
X − µ
σ
(4.7.166)
can then be written in several very useful ways:
Y =
Sn − nµ
σSn
√
n
=
Mn − µ
σMn /
√
n (4.7.167)
Furthermore, assuming that the reader already knows what is a Normal distribution N(µ, σ), we
will show later in detail because it is extremely important (!) that the probability of the random
variable ¯X, average of n identically distributed and linearly independent random variables, has
for law (obviously):
N µ,
σ
√
n
(4.7.168)
4.7.3.1.4 Coefﬁcient of Correlation
Now consider X and Y two random variables having for covariance:
cX,Y = E [(X − µX)(Y − µY )] (4.7.169)
Theorem 4.26. We have:
(cX,Y )2
≤ V (X)V (Y ) (4.7.170)
We will prove this relation immediately because the use of the covariance alone for data analysis
is not always great because it is not strictly limited and easy to use (at interpretation). We will
construct an indicator easier to use in business.
Proof 4.26.1. We choose any constant a and we calculate the variance of:
aX + Y (4.7.171)
We can then immediately write using the properties of the variance and the of the mean:
V(aX + Y ) = a2
V(X) + V(Y ) + 2acX,Y (4.7.172)

Draft
The right quantity is positive or null for any a by construction of the variance (left). So the
discriminant of the expression, seen as a polynomial in a is of the type:
P(x) = ax2
+ bx + c = a x +
b
2a
2
−
b2
− 4ac
4a2
⇒
P(a) = Va2
+ 2cX,Y a + V(Y ) = V(X) a +
2cX,Y
2V(X)
2
−
(2cX,Y )2
− 4V (X)V(Y )
4V(X)2
(4.7.173)
Because P(a) is positive for any a we have as only possibility that:
(2cX,Y )2
− 4V(X)V(Y ) ≤ 0 (4.7.174)
Therefore after simplification:
(cX,Y )2
≤ V(X)V(Y ) (4.7.175)
Q.E.D.
This gives us also:
(cX,Y )2
V(X)V(Y )
≤ 1 ⇔
|cX,Y |
V(X)V(Y )
≤ 1 (4.7.176)
Finally we get some a statistical inequality named Cauchy-Schwarz inequality:
−1 ≤
cX,Y
V(X)V(Y )
≤ 1 (4.7.177)
If the variances of X and Y are non-zero, the correlation between X and Y is defined by the
linear correlation coefficient (it is a standardized covariance so that its amplitude does not
depend on the chosen unit measure) and written:
RX,Y =
cX,Y
V(X)V(Y )
(4.7.178)
Which can also be written in an expanded form (using Huyghens theorem):
RX,Y =
E(XY ) − E(X)E(Y )
V(X)V(Y )
=
E(XY ) − E(X)E(Y )
(E(X2) − E(X)2) (E(Y 2) − E(Y )2)
(4.7.179)
or more condensed:
RX,Y =
cX,Y
σXσY
(4.7.180)
Remark
Note that normally, the letter R is reserved to say that this is an estimator of the correlation
coefficient but the definition above is not an estimator (the variances doesn’t have the
small hat...) and that, strictly speaking, we should then write ρX,Y according to the
traditions of use.

Draft
Whatever the units and the orders of magnitude, the correlation coefficient is a number between
−1 and 1 without units (so its value does not depend on the unit of measure, which is by far not
the case for all statistical indicators!). It reflects more or less the linear dependence of X and
Y or geometrically more or less the flattness magnitude. We can therefore say that a coefficient
of correlation of zero or close to 0 correlation means that there is no linear relation between the
characters. But it does not involve any notion of more general independence.
When the correlation coefficient is near 1 or −1, the characters are said to be strongly correlated.
We must be careful with the frequent confusion between correlation and causality. Thus, two
phenomena that are correlated does not imply in any way that one is the cause of the other.
Therefore:
• If RX,Y = −1 we are dealing with a pure negative correlation (in the case of a linear
relation all measurement points are located on a straight line with a negative slope).
• If −1 RX,Y 1 we are dealing with a negative or positive correlation named imper-
fect correlation (in the case of a linear relation all measurement points are located on a
straight positive or negative slope respectively).
• If RX,Y = 0 the correlation is zero... (in the case of a linear relation all the measurement
points are located on a straight line of slope zero).
• If RX,Y = 1 we are dealing with a pure positive correlation (in the case of a linear
relation all measurement points are located on a straight positive slope).
The analysis of the correlation coefficient has the objective of determining the degree of as-
sociation between variables: it is often expressed as the coefficient of determination, which is
the square of the correlation coefficient. The coefficient of determination thus measures the
contribution of a variable to the explanation of the second.
Using the expressions of mean and standard deviation of equiprobable variables as demonstrated
above (thus the idea of computing the correlation of two random variables is a good idea if they
are jointly gaussian), we start:
RX,Y =
E(XY ) − E(X)E(Y )
(E(X2) − E(X)2) (E(Y 2) − E(Y )2)
(4.7.181)
To obtain the estimator of the coefficient of correlation
RX,Y =
1
n
n
i=1
xiyi −
1
n
n
i=1
xi
1
n
n
i=1
yi

1
n
n
i=1
x2
i −
1
n2
n
i=1
xi
2



1
n
n
i=1
y2
i −
1
n2
n
i=1
yi
2


(4.7.182)
where we see that the covariance becomes the average of the products minus the product of
averages.

Draft
Thus after simplification we get a famous expression:
RX,Y =
n
i=1
xiyi −
1
n
n
i=1
xi
n
i=1
yi


n
i=1
x2
i −
1
n
n
i=1
xi
2




n
i=1
y2
i −
1
n
n
i=1
yi
2


(4.7.183)
The correlation coefficient can be calculated in the English version of Microsoft Excel 11.8346
and others with the integrated gy‚‚iv@ A function.
We will see in the section Numerical Methods a more general expression of the correlation
coefficient.
Remarks
R1. In the literature, the experimental correlation coefficient is often named sampling
Pearson coefficient (in the equiprobable case) and when we carry it to the square, then
we name it the coefficient of determination.
R2. Often the square of the coefficient is somewhat improperly interpreted as the % of
variation explained in the response variable Y by the explanatory variable X.
Finally, note that we have the following relation which is used a lot in practice (see the section
Economics for famous detailed examples!):
V(X + Y ) = V(X) + V(Y ) + 2cX,Y = V(X) + V(Y ) + 2RX,Y V(X)V(Y ) (4.7.184)
or the version with the standard deviation even more famous:
σX+Y = σ2
X + σ2
Y + 2RX,Y σXσY (4.7.185)
It is a relation that we can often see in finance in the calculation of the VaR (Value at Risk)
according to RiskMetrics methodology proposed by JP Morgan (see section Economy).
Let us see a small application example of the correlation but that has nothing to do with VaR (at
least for the moment...).

Draft
Example:
An airline company has 120 seats available that she reserves for connecting pas-
sengers from two flights arrived earlier in the journey and that have to go to Frankfurt.
The first flight arrived from Manila and the number of passengers on board follows a
Normal distribution with mean 50 and variance 169. The second flight arrives in Taipei
and the number of passengers on board follows a Normal distribution with mean 45 and
variance 196.
The linear correlation coefficient between the number of passengers of both flights was
measured as:
RX,Y = 0.5 (4.7.186)
The law that follows the number of passengers for Frankfurt if we assume that the law of
the couple also follows a Normal distribution (according to statement!) is:
X + Y = N (µY + µX, σX+Y ) (4.7.187)
with:
µY + µX = 50 + 45 = 95
σX+Y = σ2
X + σ2
Y + 2RX,Y σXσY
(4.7.188)

Draft
RX,Y = 0.5 (4.7.189)
The law that follows the number of passengers for Frankfurt if we assume that the law of
the couple also follows a Normal distribution (according to statement!) is:
X + Y = N (µY + µX, σX+Y ) (4.7.190)
with:
µY + µX = 50 + 45 = 95
σX+Y = σ2
X + σ2
Y + 2RX,Y σXσY = 169 + 196 + 2 · 0.5
√
169 · 196 ∼= 23.38
(4.7.191)
This is a bad start for customer satisfaction in the long term...
4.7.3.2 Continuous Variables and Moments
Definitions:
1. We say that X is a continuous variable if its cumulative distribution function C.D.F.
is continuous (already defined above). The distribution function of X is defined by for
x ∈ R or a truncated subset of R:
F(x) = P(X ≤ x) (4.7.192)
that is the cumulative probability that the random variable X is smaller than or equal to
the set value x. We also have of course:
0 ≤ P(X) ≤ 1 (4.7.193)
2. We denote by:
G(x) = 1 − F(x) = P(X x) (4.7.194)
the survival function or tail function.
3. If furthermore the distribution function F of X is continuously differentiable of derivative
f (or sometimes denoted by ρ) named density function or mass function or just simply
distribution function then we say that X is absolutely continuous and in this case we
have:
P(x1 ≤ X ≤ x2) =
x2ˆ
x1
f(x)dx = F(x2) − F(x1) (4.7.195)

Draft
with the normalization condition:
P(X −∞) =
+∞ˆ
−∞
f(x)dx =
+∞ˆ
−∞
dF(x) = 1 (4.7.196)
Any probability distribution function must satisfy the integral of normalization in its do-
main of definition!
Remark
It is interesting to note that the definition implies that the probability that a completely
continuous random variable takes a given value tends to zero! So it is not because an
event has almost a zero probability that it can not happen!!!
The average being defined by a sum weighted by probabilities for a discrete variable, it becomes
an integral for a continuous variable:
E(X) =
+∞ˆ
−∞
xf(x)dx (4.7.197)
and therefore the variance is written as:
V(X) =
+∞ˆ
−∞
[x − E(X)]2
f(x)dx (4.7.198)
Then we have also the median that is logically redefined in the case of a continuous random
variable by:
P(X ≤ Me) = P(X ≥ Me) =
Meˆ
−∞
f(x)dx =
+∞ˆ
Me
f(x)dx =
1
2 (4.7.199)
and it rarely coincides with the average!
And the modal value is given by the value of x where:
df(x)
dx
= 0 (4.7.200)
Statisticians often use the following notations for the expected mean of a continuous variable:
E(X), M(X), µX, µ (4.7.201)
and for the variance:
V(X), S(X), σ2
X, σ2
(4.7.202)
That is the same as for the moment of discrete variable.
Thereafter, we will calculate these different moments indicators with detailed proofs only for
the most used cases.

Draft
4.7.4 Fundamental postulate of statistics
One of the ultimate goals of statistics is, starting from a sample, to find the analytical distribution
function that gave birth to the sample. This goal will be presented on this web site as a postulate
(although this assumption is very difficult to apply in practice).
Postulate: For any empirical distribution function ˆFn(x) of the n-th measurement of the x
random variable we can associate a theoretical distribution function F(x) to which it converges
when the sample size is large enough if:
Xn = sup | ˆFn(x) − F(x)| (4.7.203)
is the random variable defined as the largest difference (in absolute value) between ˆFn(x) and
F(x) (observed for all values of x for a given sample), then Xn converges to 0 almost surely.
Remark
Mathematicians of Statistics prove this postulate rigorously as a theorem named the fun-
damental theorem of statistics or the Glivenko-Cantelli theorem regarding continuous
functions. Personally, even if we offends the experts, we think that this proof is not one
because because it is very far away from the practical reality (yes this is our physicist
side that emerges...) and this theoretical result leads many practitioners do their utmost
(excluding data, transformations and other abominations) to find a known distribution
law that they can adjust to their measured data.
4.7.5 Diversity Index
It happens in the field of biology or business that you it is asked to a statistician or analyst to
measure the diversity of a number of predefined elements. For example, imagine a multinational
with a range of well-defined products and some of the stores (customers) in the world can choose
a subset of this range for their business sales. The request is then to make a ranking of stores
that sell the widest range of branded products and that by taking also into account the quantity.
For example, we have a list of a total 4 products in our catalog. By hazard, three of our cus-
tomers sell our 4 products but we would like to know which customers sells the greatest diversity
and this by taking into account the quantities.
We have the following sales data by product for the customer 1:
Customer 1
Product 1 5
Product 2 5
Product 3 5
Product 4 5

Draft
For the customer 2:
Customer 1
Product 1 1
Product 2 1
Product 3 1
Product 4 17
and for the customer 3:
Customer 1
Product 1 2
Product 2 2
Product 3 2
Product 4 34
A measure of information (diversity of states) that is well suited to this purpose is the Shannon
formula introduced in the section of Statistical Mechanics whose mean:
S(x) = E(h(x)) = −λ
n
i=1
pi log(pi) (4.7.204)
Arbitrarily, we will take and the logarithm in base 10 (so, if we have 10 equiprobable variables,
entropy is unitary for example...).
Therefore we have:
S(x) = −
n
i=1
pi log10(pi) (4.7.205)
We will rewrite this more adequately for the application in business. Thus, if n is the number
of products and pi the proportion (or relative frequency) of sales of product i from all sales N
then:
pi =
fi
N
(4.7.206)
Then we have:
S(x) = −
n
i=1
fi
N
log10(
fi
N
) = −
1
N
n
i=1
fi [log10(fi) − log10(N)]
=
n
i=1
fi [log10(N) − log10(fi)]
N
=
log10(N)
n
i=1
fi −
n
i=1
fi log10(fi)
N
=
N log10(N) −
n
i=1
fi log10(fi)
N
(4.7.207)

Draft
This gives for the customer 1 (we stay in base 10 for the logarithm):
S1 =
N log(N) −
n
i=1
fi log(fi)
N
=
20 log(20) − (5 log(5) + 5 log(5) + 5 log(5) + 5 log(5))
20
=
20 log(20) − 20 log(5)
20
= log(20) − log(5) = log(4) = 0.602
(4.7.208)
which is the maximum possible value (each state is equally likely). And for customer 2 we
have:
S2 =
N log(N) −
n
i=1
fi log(fi)
N
=
20
= 0.255
(4.7.209)
And finally for customer 3:
S3 =
N log(N) −
n
i=1
fi log(fi)
N
=
40
= 0.255
(4.7.210)
Thus, the customer that has the greatest diversity is the first one. We also see an interesting
property of the Shannon formula with customer 2 and 3 and this is that the quantity does not
affect diversity (since the only difference between the two customers is that the quantity is
multiplied by a factor of 2 and not diversity)!
4.7.6 Distribution Functions (probabilities laws)
When we observe probabilistic phenomena, and we take note of the values taken by them and
that we report them graphically, we can observe that the individual measurements follow a
typical characteristic which is sometimes adjustable theoretically with a good level of quality.
In the field of probabilities and statistics, we call these characteristics distribution functions
because they indicate the frequency with which the random variable appears for given values.
Remark
We sometimes simply use the term function or law to describe these characteristics.
These functions are in practice bounded by what we name the range of distribution which is
the difference between the maximum value (on the right) and the minimum value (on the left)
of the observed values:
R = max − min (4.7.211)

Draft
In theory they are not necessarily bounded and then we talk (see section Functional Analysis)
about a domain of definition or more simply about the support of the function.
If the observed values are distributed in a certain way then there is a probability (or cumulative
probability in the case of continuous distribution functions) to have a certain value of the
distribution function.
In industrial practice (see section Industrial Engineering), the range of statistical values is im-
portant (as well as the standard deviation) because it gives an indication of the variation of a
process (variability).
If L denote any possible univariate distribution function the range of the function is simply
denoted by L if its domain of definition is R otherwise if it is bounded you will typically see
something like L]a,b].
Definitions:
D1. The mathematical relation that gives the probability of a given value of the distribution
function a random variable is named the density function (or probability density func-
tion), mass function or marginal function.
D2. The mathematical relation that gives the cumulative probability that a random variable to
be lower than or equal to a certain value of the distribution function is referred to as the
repartition function or cumulative function or cumulative distribution function.
D3. Random variables are independent and identically distributed (i.i.d.) if they all follow
the same distribution function, with the same parameters values and that they are inde-
pendent.
uSch functions are very numerous, we offer then here to the reader a detailed study of the most
known only.
Before going any further it could be useful to know that if X is a continuous or discrete random
variable, then are several tradition of notation in the literature to indicate that it follows a given
probability distribution L. Here are the most common:
X ∼ L
X
L
= L
X → L
X = L
(4.7.212)
In this section and throughout the book in general, we will use the last notation!
Here is the list of the distribution functions that we will see here as well as distribution functions
commonly used in the industry and located in other chapters/section and those whose proof has
yet still to be written:
• Discrete Uniform Distribution U(a, b) (see below)
• Bernoulli Distribution B(1, p) (see below)

Draft
• Geometric Distribution G(N) (see below)
• Binomial Distribution B(N, k) (see below)
• Binomial Negative Distribution NB(N, k, p) (see below)
• Hypergeometric Distribution H(n, p, m, k) (see below)
• Multinomial Distribution (see below)
• Poisson Distribution P(µ, k) (see below)
• Gauss-Laplace/Normal Distribution N(µ, σ2
) (see below)
• Log-Normal Distribution (see below)
• Continuous Uniform Distribution (see below)
• Triangular Distribution (see below)
• Pareto Distribution (see below)
• Exponential Distribution (see below)
• Weibull Distribution (see section Industrial Engineering)
• Generalized Exponential Distribution (see section Numerical Methods)
• Erlang/Erlang-B/Erlang-C Distributions (see section Quantitative Management)
• Cauchy Distribution (see below)
• Beta Distribution (below and section Quantitative Management)
• Gamma Distribution (see below)
• Chi-2 Distribution (see below)
• Student Distribution (see below)
• Fisher-Snedecor Distribution (see below)
• Benford Distribution (see below)
• Logistic Distribution (see section Numerical Methods)
• Square Gauss distribution (still must be written)
• Extreme value distribution (still must be written)
Remark
The reader will ﬁnd the mathematical developments of the Weibull distribution function
in the section on Industrial Engineering (Engineering chapter), and the logistic distribu-
tion function in the section of Numerical Methods.
4.7.6.1 Discrete Uniform Distribution
If we accept that it is possible to associate a probability to an event, we can conceive of situations
where we can assume a priori that all elementary events are equally likely (that is to say, they
have the same probability to occur). We then use the ratio between the number of favorable
cases and the number of possible cases to calculate the probability of all events in the Universe
of events U. More generally, if U is a ﬁnite set of equally likely events and A is part of U, then
we have using set theory notation (see section Set Theory):
P(A) =
Card(A)
Card(U)
=
#A
#U
(4.7.213)
More commonly, if e is an event that may have N equally likely possible outcomes. Then the
probability of observing the outcome of this given event follows a discrete uniform function

Draft
(or discrete uniform law) given by the relation:
Pe =
1
N
(4.7.214)
Whose mean (or average) is given by:
E(X) =
i
pixi =
i
Pexi = Pe
i
xi =
1
N i
xi (4.7.215)
If we put ourselves in the particular case where xi = 1 with i = 1...N. We then have (see
Sequences And Series):
E(X) =
1
N i
xi =
1
N
N
i=1
i =
1
N
N(N + 1)
2
=
N + 1
2
(4.7.216)
If the random variable e take all values between [a, b] (another special case) such the distribution
will be now denoted by U(a, b) then it should be obvious that we have for the expected mean:
E(X) =
b
i=a
Pei = Pe
b
i=a
i =
1
b − a + 1
b
i=a
i =
1
b − a + 1
b
i=1
i −
a−1
i=1
i
=
1
b − a + 1
b(b + 1)
2
+
(a − 1)((a − 1) + 1)
2
=
1
b − a + 1
b(b + 1) − a(a − 1)
2
=
1
b − a + 1
(b − a + 1)(b + a)
2
=
a + b
2
(4.7.217)
For the variance we have (always using the results of the section on Sequences and Series):
V(X) =
N
i=1
pi(i − µ)2
=
N
i=1
Pe(i − µ)2
=
N
i=1
1
N
(i − µ)2
=
1
N
N
i=1
(i − µ)2
=
1
N
N
i=1
i2
− 2µ
N
i=1
i +
N
i=1
µ2
=
1
N
N
i=1
i2
− 2µ
N
i=1
i + Nµ2
=
1
N
N
i=1
i2
−
2
N
µ
N
i=1
i + µ2
=
1
N
N(N + 1)(2N + 1)
6
−
2
N
N + 1
2
N
i=1
i +
N + 1
2
2
=
(N + 1)(2N + 1)
6
−
N + 1
N
N
i=1
i +
(N + 1)2
4
=
(N + 1)(2N + 1)
6
−
N + 1
N
N(N + 1)
2
+
(N + 1)2
4
=
(N + 1)(2N + 1)
6
−
(N + 1)2
2
+
(N + 1)2
4
=
(N + 1)(2N + 1)
6
−
(N + 1)2
4
=
2(N + 1)(2N + 1) − 3(N + 1)2
12
=
2(2N2
+ N + 2N + 1) − 3N2
− 6N − 3
12
=
4N2
+ 6N + 2 − 3N2
− 6N − 3
12
=
N2
− 1
12
(4.7.218)

Draft
If the random variable e take all values between [a, b] (another special case) such the distribution
will be now denoted by U(a, b) then it should be obvious that we have for the variance:
V(X) =
N2
− 1
12
=
(b − a + 1)2
+ 1
12
(4.7.219)
By symmetry of the distribution if all values of the domain of deﬁnition [a, b] are taken by the
random variable we have for the median:
Me = E(X) =
a + b
2
(4.7.220)
Here is an plot example of the mass distribution function and cumulative distribution function
respectively for discrete uniform law of parameters {1, 5, 8, 11, 12} (we see that each value is
equally likely):
Figure 4.70 – Uniform law U (density and cumulative distribution function)
As we can see in the above diagram the cumulative distribution function can be written:
F(x) = P(X ≤ x) =
#(i : xi ≤ x)
N
(4.7.221)
Remark
For sure the discrete uniform distribution has no speciﬁc modal value M0!

Draft
4.7.6.2 Bernoulli Distribution
If we are dealing with a binary observation then the probability of an event is constant from
one observation to the other if there is no memory effect (in other words: a sum of Bernoulli
variables, two by two independent).
We name this kind of observations where the randoms variables takes the values 0 (false) or 1
(true), with probability q = 1 − p respectively p, Bernoulli trials with contrary events with
contrary probabilities.
Thus, a random variable X follows a Bernoulli function B(1, p) (or Bernoulli law)if it can
take only the values 0 or 1 , associated with probabilities p and q and so that q + p = 1 and:
P(X = 0) = q P(X = 1) = p = 1 − q (4.7.222)
The classic example of such a process is the game of piece face or sampling with replacement
or be considered as such (this last case is very important in industrial practice). There certainly
is no need for the reader to formally verify that the cumulative probability is unitary...
Remark
The introduction above is perhaps not relevant for business, but we will see in the section
of Quantitative Techniques that the Bernoulli function naturally appears at the beginning
of our study of queuing theory.
Note that, by extension, if we consider N events where we get in a particular order k times one
possible outcomes (success) and the other N − k (fail) times, then the probability of such a
series (k successes and N − k failures ordered in any particular way) is given by :
P(N, k) = pk
(1 − p)N−k
= pk
qN−k
(4.7.223)
with N ∈ N∗
according to what we got during the study of combinatorics in the section of
Probabilities!
Here is an example plot of the cumulative distribution function for q = 0.3:
Figure 4.71 – Bernoulli law B (cumulative distribution function)

Draft
The Bernoulli function has therefore for expected mean (average) choosing p as the probability
of the event of interest:
µ = E(X) =
i
pixi = p · 1 + (1 − p) · 0 = p (4.7.224)
and for variance (we use the Huygens theorem proved above):
V(X) = σ2
= E(X2
) − E(X)2
= p − p2
= p(1 − p) = pq (4.7.225)
The modal value M0 of the Bernoulli law depends on the values of p or q. So we have (it could
be obvious for the reader):
M0 = 0 ⇔ p q
M0 = {0, 1} ⇔ p = q
M0 = 1 ⇔ q 0
(4.7.226)
Remark
For sure the Bernoulli distribution has no specific median value Me!
4.7.6.3 Geometric Distribution
The geometric law G(N) or Pascal’s law consist in a Bernoulli trial, where the probability of
success is p and that of failure q = 1 − p are constant, that we renew independently until the
first success.
Remember that during our presentation of the Bernoulli law we have deduce an extension to N
such that:
P(N, k) = pk
(1 − p)N−k
= pk
qN−k
(4.7.227)
Therefore the probability to get the first success k = 1 after N trials is:
G(N) = p(1 − p)N−1
= pqN−1
(4.7.228)
with N ∈ N∗
.
As you can see, greater is N, smaller is the probability G(N). This can be seem non-logic but
in fact it is! Indeed in the sentence the probability to get the first success after N trials, you
must not forget that it is written after and not during.
Therefore for sure... the probability to have N −1 failures followed by 1 success will be always
be smaller when N increase (have a look the figure a little bet further below for p = 0.5 can
help to understand).

Draft
This law has for expected mean:
µ = E(X) =
i
pixi =
+∞
N=1
pqN−1
N =
+∞
N=1
p(1 − p)N−1
N = p
+∞
N=1
(1 − p)N−1
N (4.7.229)
However, the last relation can also be written:
+∞
N=1
NqN−1
=
1
(1 − q)2
(4.7.230)
Indeed, we proved in the section of Sequences and Series during our study of geometric series
that:
n
k=0
qk
=
1 − qn+1
1 − q
(4.7.231)
Taking the limit n → +∞ when we get:
+∞
k=0
qk
=
1
1 − q
(4.7.232)
because 0 ≤ q 1. Then we just derivate both members of equality with respect to q and we
get:
+∞
k=1
kqk−1
=
1
(1 − q)2
(4.7.233)
This done let us continue...
We have then the average number of trials X it takes to get the ﬁrst success (or in other words,
the expected rank - number of expected trials - to see the ﬁrst success):
E(X) =
+∞
N=0
NP(X = N) =
+∞
N=0
NpqN−1
=
p
(1 − q)2
=
p
p2
=
1
p
(4.7.234)
Now we calculate the variance and reminding once again (Huygens theorem):
V(X) = E(X2
) − E(X)2
(4.7.235)
So let’s start by calculating E(X2
):
E(X2
) =
+∞
N=0
N2
P(X = N) = p
+∞
N=0
N2
qN−1
= p
+∞
N=0
N(N − 1 + 1)qN−1
= p
+∞
N=1
N(N − 1)qN−1
+ p
+∞
N=0
NqN−1
(4.7.236)
The last term of this expression is equivalent to the expected mean calculated previously. Thus:
p
+∞
N=0
NqN−1
=
1
p
(4.7.237)

Draft
It remains to calculate:
p
+∞
N=1
N(N − 1)qN−1
(4.7.238)
We have:
p
+∞
N=1
N(N − 1)qN−1
= pq
+∞
N=2
N(N − 1)qN−2
(4.7.239)
But deriving the following equality:
+∞
k=1
kqk−1
=
1
(1 − q)2
(4.7.240)
We get:
+∞
k=1
k(k − 1)qk−2
=
2
(1 − q)3
(4.7.241)
Therefore:
pq
+∞
N=2
N(N − 1)qN−2
= pq
2
(1 − q)3
= pq
2
p3
=
2q
p2
(4.7.242)
Thus:
E(X2
) =
2q
p2
+
1
p2
(4.7.243)
Finally when it comes to ranking the expected variance of the first success (i.e.: the variance
expected number before the first successful trials):
V(X) = σ2
= E(X2
) − E(X)2
=
2q
p2
+
1
p
−
1
p2
=
1 − p
p2
(4.7.244)
The modal value is easy to get because we need to find the value of N that maximize the
definition of the geometric law:
pqN−1
(4.7.245)
and we hope that it is immediate to the reader that this is satisfy when N = 1 therefore:
M0 = pq1−1
= pq0
= p (4.7.246)
Now let us determine the median Me to finish. For this, by definition we know we must have:
Me
N=1
pqN−1
=
+∞
N=Me
pqN−1
= 0.5 (4.7.247)

Draft
But we can rewrite:
Me
N=1
pqN−1
=
+∞
N=Me
pqN−1
= pqMe−1
+∞
N=0
qN
= pqMe−1 1
1 − q
= pqMe−1 1
p
= qMe−1
= 0.5
(4.7.248)
Therefore (in base 10):
log qMe−1
= (Me − 1) log(q) = log(0.5) (4.7.249)
Finally base on our definition of the median we get:
Me =
log(0.5)
log(q)
+ 1 (4.7.250)
Now we determine the cumulative function of the geometrical law. We start from:
G(N) = pqN−1
(4.7.251)
Then we have by definition the cumulative probability of that the experience is successful in the
first N trials:
P(X ≤ N) = 1 −
+∞
j=N+1
pqj−1
= 1 − p
+∞
j=N+1
qj−1
(4.7.252)
with N being for sure an integer of values 0, 1, 2, .... We write:
j − 1 = n + k ⇒ k = n − j + 1 (4.7.253)
We then have for the CDF:
P(X ≤ N) = 1 − p
+∞
k=0
qN+k
= 1 − p
+∞
k=0
qN
qk
= 1 − pqN
+∞
k=0
qk
= 1 − pqN
+∞
k=0
qk
= 1 − pqN 1
1 − q
= 1 − (1 − q)qN 1
1 − q
= 1 − qN
(4.7.254)
Example:
You try late at night and in the dark, to open a lock with a bunch of five keys, without
attention, because you are a little tired (or a little tipsy ...) you will try each key.
Knowing that only one key will work, what is the probability of using the right key at
the N-th test?
The solution is:
G(N) = pqN−1
=
1
5
4
5
N−1
(4.7.255)

Draft
Plot of the mass function and cumulative distribution function for the Geometric distribution
with parameter p = 0.5:
Figure 4.72 – Geometric law G (mass and cumulative distribution function)
4.7.6.4 Binomial Distribution
We come back now to our Bernoulli experiment. More generally, any particular N-tuple con-
sisting of k successes and of N − k failures will have for probability (within a sampling with
replacement or without replacement if the population is large ... in a first approximation):
P(N, k) = pk
(1 − p)N−k
= pk
qN−k
(4.7.256)
to be drawn (or appear) whatever the order of appearance of successes and failures (the reader
will have perhaps notice that this is a generalization of the geometric distribution, just write
k = 1 to find the geometric distribution back).
But we know that the combinatorial determines the number of N-tuples of this type (the number
of ways to order the appearance of failures and successes). The number of possible arrange-
ments is, as we proved it (see section Probabilities), given by the binomial coefficient (we recall
that the notation in this book does not comply with ISO standard 31-11):
CN
k =
N!
k!(N − k)!
(4.7.257)
So as the probability of obtaining a given series of k successes and N − k failures is always the
same (regardless of the order) then we have just to multiply the probability of a particular series
by the binomial coefficient (this is equivalent to a sum ) such that:
P(N, k) = CN
k P =
N!
k!(N − k)!
pk
(1 − p)N−k
=
N!
k!(N − k)!
pk
qN−k
(4.7.258)

Draft
to get the total probability to obtain any of these possible series (since each is possible).
Remark
This is equivalent to the study of a sampling with simple replacement (see Probabilities)
with constraint on the order or to the study of a series of successes and failures. We will
use this relation in the context of the queuing theory or reliability (see section Industrial
Engineering). Note that in the case of large populations, even if the sampling is not with
replacement it can be considered as with...
Written in another way this gives the binomial function (or binomial law), also known as
the following distribution function:
B(N, k) = CN
k pk
(1 − p)N−k
= CN
k pk
qN−k
=
N
k
pk
qN−k
(4.7.259)
and sometimes also denoted by β(n, p) with a lowercase n or uppercase N (it does not really
matter...) and can be calculated in the English version of Microsoft Excel 11.8346 using the
fsxywhsƒ„@ A function.
We sometimes say that the binomial law is not exhaustive as the size of the initial population is
not apparent in the expression of the law.
Remark
The Binomial distribution is named Symmetric Binomial Distribution when p = 0.5.
Example:
We want to test the alternator of a generator. The probability of failure at solicitation of
this material is estimated to be 1 failure per 1, 000 starts.
We decided to test 100 starts. The probability of observing one failure in this test is:
B(N = 100, k = 1) = CN
k pk
(1 − p)N−k
=
N!
k!(N − k)!
=
100!
1!(100 − 1)!
1
1000
1
1 −
1
1000
99
∼= 9%
(4.7.260)
We obviously have for the cumulative distribution function (very useful in practice for suppliers
batch control or reliability as we will see in the section of Industrial Engineering!):
N
k=0
CN
k pk
(1 − p)N−k
= 1 (4.7.261)
Indeed, we have proved in the section of Calculus the binomial theorem:
(x + y)n
=
n
k=0
Cn
k xk
yn−k
) = 1 (4.7.262)

Draft
Therefore:
N
k=0
CN
k pk
(1 − p)N−k
= (p + (1 − p))N
= 1N
= 1 (4.7.263)
Instead of calculating such cumulated probability rather than hand it is better to use Microsoft
Excel 11.8346 (or any other widely known software) with the function g‚s„fsxyw@A to not
bother to calculate these type of values.
The expected mean (average) of B(N, k) is given by:
E(X) =
N
k=0
pik =
N
k=0
P(X = k)k =
N
k=0
CN
k pk
(1 − p)N−k
k =
N
k=1
CN
k pk
(1 − p)N−k
k
(4.7.264)
But having:
CN
k =
N
k
CN−1
k−1 ⇔ kCN
k = NCN−1
k−1 (4.7.265)
We ﬁnally get:
E(X) =
N
k=1
CN
k pk
(1 − p)N−k
= N
N
k=1
CN−1
k−1 pk
(1 − p)N−k
= Np
N
k=1
CN−1
k−1 pk−1
(1 − p)(N−1)−(k−1)
= Np
N−1
k=0
CN−1
k pk
(1 − p)(N−1)−k
= Np(p + (1 − p))N−1
= Np
(4.7.266)
that gives the average number of times that we will get the desired outcome of probability p
after N trials.
The mean of the binomial distribution is sometimes noted in the specialized literature with the
following notation if r is the potential number of possible expected outcomes in a population of
size n:
E(X) = Np = N
r
n
(4.7.267)
Before calculating the variance, we need to introduce the following equality:
N−1
s=0
s
(N − 1)!
s!(N − 1 − s)!
ps
qN−1−s
= (N − 1)p (4.7.268)

Draft
Indeed, let us proof this relation using the previous developments:
N−1
s=0
s
(N − 1)!
s!(N − 1 − s)!
ps
qN−1−s
=
N−1
s=0
s
(N − 1)!
s(s − 1)!(N − 1 − s)!
ps
qN−1−s
=
N−1
s=1
(N − 1)!
(s − 1)!(N − 1 − s)!
ps
qN−1−s
= (N − 1)
N−1
s=1
(N − 2)!
(s − 1)!(N − 1 − s)!
ps
qN−1−s
= (N − 1)
N−2
j=0
(N − 2)!
j!(N − 2 − j)!
pj+1
qN−2−j
= (N − 1)p
N−2
j=0
CN−2
j pj
qN−2−j
= (N − 1)p
N−2
j=0
CN−2
j pj
qN−2−j
(4.7.269)
We recognize in the last equality the cumulative distribution function that is equal to 1. There-
fore:
(N − 1)p
N−2
j=0
CN−2
j pj
qN−2−j
= (N − 1)p · 1 = (N − 1)p (4.7.270)
We start now the (long) calculation of the variance of the binomial distribution by using the

Draft
previous results:
V(k) =
N
k=0
pi(k − µ)2
=
N
k=0
N!
k!(N − k)!
pk
qN−k
(k − Np)2
=
N
k=0
N!
k!(N − k)!
pk
qN−k
(k − µ)2
=
N
k=0
k2 N!
k!(N − k)!
pk
qN−k
+
N
k=0
µ2 N!
k!(N − k)!
pk
qN−k
−
N
k=0
2kµ
N!
k!(N − k)!
pk
qN−k
=
N
k=0
k2 N!
k!(N − k)!
pk
qN−k
+ µ2
N
k=0
N!
k!(N − k)!
pk
qN−k
− 2µ
N
k=0
k
N!
k!(N − k)!
pk
qN−k
=
N
k=0
k2 N!
k!(N − k)!
pk
qN−k
+ µ2
N
k=0
CN
k pk
qN−k
=1
−2µ
N
k=0
k
N!
k!(N − k)!
pk
qN−k
=µ=Np
=
N
k=0
k2 N!
k!(N − k)!
pk
qN−k
+ µ2
− 2µ2
=
N
k=0
k2 N!
k!(N − k)!
pk
qN−k
− µ2
=
s=k−1
−µ2
+ N
N−1
s=0
(s + 1)
(N − 1)!
s!(N − 1 − s)!
ps
qN−(s+1)
= −µ2
+ Np
N−1
s=0
(s + 1)
(N − 1)!
s!(N − 1 − s)!
ps
qN−s−1
= −µ2
+ Np
N−1
s=0
s
(N − 1)!
s!(N − 1 − s)!
ps
q(N−1)−s
+ Np
N−1
s=0
s
(N − 1)!
s!((N − 1 − s)!
ps
q(N−1)−s)
= −µ2
+ Np
N−1
s=0
sCN−1
s ps
q(N−1)−s
+ Np
N−1
s=0
sCN−1
s ps
q(N−1)−s)
= −µ2
+ Np(N − 1)p + Np
N−1
s=0
sCN−1
s ps
q(N−1)−s)
= −µ2
+ Np2
(N − 1) + Np · 1 = −µ2
+ Np2
(N − 1) + Np
= −µ2
+ N2
p2
− Np2
+ Np = −µ2
+ µ2
− Np2
+ Np = Np(1 − p)
= Npq
(4.7.271)
Finally:
V(k) = σ2
= Np(1 − p) = Npq (4.7.272)
The standard deviation of the binomial distribution is sometimes noted in the specialized liter-
ature in the following way if r is the potential number of expected outcomes in a population of
size n and s the not expected one:
σ = Npq = N
r
n
s
n
=
1
n
√
Nrs (4.7.273)
Here is a plot example of the binomial B(10, 0.5) distribution and cumulative distribution func-
tion:

Draft
Figure 4.73 – Binomial law B (mass and cumulative distribution function)
It could be useful to note that some employees in companies normalize the calculation of the
mean and standard deviation to the unit of N. Then we have:
E
k
N
=
Np
N
= p
V
k
N
=
1
N2
V(X) =
1
N2
Npq =
1
N
p(1 − p) ⇔ σ =
p(1 − p)
N
(4.7.274)
Example:
In a sample of 100 workers, 25% are late at least once a week. The mean number and
variance of late people is then:
E(k) = Np = 100 · 0.25 = 25
σk = Np(1 − p) = 100 · 0.25(1 − 0.25) ∼= 4.33
(4.7.275)
Normalized to the unit of N this give us:
E
k
N
= p = 0.25
σk/N =
p(1 − p)
N
(4.7.276)
Let us now calculate the mode. Because the function is discrete we can not use derivative. Then
we will use a hint. We compute the ratio:
B(N, k)
B(N, k + 1)
(4.7.277)
and we check that this ratio is 1 for every k k∗
and ≤ 1 for every k ≥ k∗
, for some integer
k∗
that is the k value corresponding to the modal value.

Draft
Let ak = P(X = k). We have:
ak = CN
k pk
qN−k
ak+1 = CN
k+1pk+1
qN−k−1
(4.7.278)
We calculate the ratio
ak+1
ak
. Note that:
ak+1
ak
=
CN
k+1pk+1
qN−k−1
CN
k pkqN−k
=
N!
(k + 1)!(N − (k + 1))!
N!
k!(N − k)!
pk+1
qN−k−1
pkqN−k
=
n − k
k + 1
p
q
=
n − k
k + 1
p
1 − p
(4.7.279)
What is important now is to analyze:
n − k
k + 1
p
1 − p
=
np − kp
k − kp + 1 − p
(4.7.280)
depending on the value of k. First we can see that this ratio is equal to 1 and therefore we have
to modes if:
np − kp
k − kp + 1 − p
= 1 ⇔ np − kp = k − kp + 1 − p (4.7.281)
That is to say if k = np + p − 1 = p(n + 1) − 1. This can be seen as the limit point of interest.
But don’t forget we are looking for the k such that the ratio is less than 1. So we try two values:
k = [p(n + 1) − 1] + 1
k = [p(n + 1) − 1] − 1
(4.7.282)
Injecting this in our ratio we see that
k = [p(n + 1) − 1] + 1 = k∗
= M0 (4.7.283)
Is the value we were looking for. Finally there are two possible values for the modes. A unique
modal value and a double modal value.
As we know the median value, is the value of X such that we have:
X
k=0
CN
k pk
(1 − p)N−k
= 0.5 (4.7.284)
But we did not yet found an easy proof to determine Me in the general case for the Binomial
law.
To conclude on the binomial law, we will develop now a result that we will need to build the
McNemar paired test for a square contingency table (and as it is squared it is also dichotomous)
that we willl study in the section of Numerical Methods.
We need for this test to calculate the covariance of two-paired binomial random variables (this
is why the covariance is non-zero):
cov(ni, nj) = E(ninj) − E(ni)(nj) (4.7.285)

Draft
As they are paired, this means that:
ni + nj = n
pi = 1 − pj
1 − pi = pj
(4.7.286)
And therefore:
cov(ni, nj) = E(ninj) − E(ni)(nj) = E(ninj) − (npi)(npj) = E(ninj) − n2
pj(1 − pj)
(4.7.287)
Now comes the difﬁculty that is to calculate E(ninj). To calculate this term it does not exist to
our knowledge other methods than looking for the law of the pair (sometimes we can get around
such approach). In this case it is a multinomial distribution (more precisely: trinomial) that it is
customary to write in the following way by construction:
M(n, k, l, pi, pj, 1 − pi − pj) =
n!
ni!nj!(n − ni − nj)!
pni
i p
nj
j (1 − pi − pj)n−ni−nj
(4.7.288)
that we will write now temporarily as following to condense the expression:
M(n, k, l, p, q, r) =
n!
k!l!(n − k − l)!
pk
ql
rn−k−l
(4.7.289)
So we have a trinomial law as we are looking for the number of times we have the event k, the
event l and neither one nor the other (so the rest of the time).
We then get:
E(kl) =
k,l
klP(kl) =
k,l
n!
k!l!(n − k − l)!
pk
ql
rn−k−l
= 0
k=0,l=0
+ 0
l=0,k=0
+ 0
k=0,l=0
+
k≥1,l≥1
kl
n!
k!l!(n − k − l)!
pk
ql
rn−k−l
(4.7.290)
If k ≥ 1 and l ≥ 1, we obtain:
kl
n!
k!l!(n − k − l)!
=
n!
k(k − 1)!l(l − 1)!(n − k − l)!
=
n(n − 1)(n − 2)!
(k − 1)!(l − 1)!(n − k − l)!
= n(n − 1)
(n − 2)!
(k − 1)!(l − 1)!(n − k − l)!
(4.7.291)
Now we use this relation in the joint mean:
E(kl) =
k≥1,l≥1
kl
n!
k!l!(n − k − l)!
pk
ql
rn−k−l
= n(n − 1)
1≤k≤n,1≤l≤n
kl
(n − 2)!
k(k − 1)!l(l − 1)!(n − k − l)!
pk
ql
rn−k−l
= n(n − 1)pq
1≤k≤n,1≤l≤n
(n − 2)!
(k − 1)!(l − 1)!(n − k − l)!
pk−1
ql−1
rn−k−l
(4.7.292)

Draft
Consider now the special case where n is equal 2. We then have:
1≤k≤n,1≤l≤n
(n − 2)!
(k − 1)!(l − 1)!(n − k − l)!
pk−1
ql−1
rn−k−l
=
1≤k≤2,1≤l≤2
0!
(k − 1)!(l − 1)!(2 − k − l)!
pk−1
ql−1
r2−k−l
=
0!
(1 − 1)!(1 − 1)!(2 − 1 − 1)!
p1−1
q1−1
r2−1−1
= 1
(4.7.293)
where the sum is reduced to only one term because if we take for example k = 2, l = 1 we get
a negative factorial at the denominator.
For n equal 3, the result will be also 1, and so on (we will assume to simplify... that some
numerical examples will sufﬁce to convince the reader of the generality of this property because
it is very boring to write with LATEX).
Then we have:
E(kl) = n(n − 1)pq
1≤k≤n,1≤l≤n
(n − 2)!
(k − 1)!(l − 1)!(n − k − l)!
pk−1
ql−1
rn−k−l
=1
= n(n − 1)pq
(4.7.294)
So in the end:
cov(ni, nj) = E(ninj) − n2
pj(1 − pj) = n(n − 1)pipj − n2
pipj = −npipj (4.7.295)
And this is the major result we will need for the study of the McNemar test.
4.7.6.5 Negative Binomial Distribution
The negative binomial distribution is applied in the same situation as for binomial distribution,
but it gives the probability to have E failures before the R-th success when the probability of
success is p (or, at contrary, the probability to have R success before E-th failure when the
failure probability is p).
We will introduce this important distribution with an example. Consider this for this purpose
the following probabilities:
P(success) = 0.2 = p P(failure) = 0.8 = 1 − p = q (4.7.296)
Imagine that we have done 10 trials and we wanted to stop at the third success and that the 10th
trial is the third successful one! We will write this:
[1 2 3 4 5 6 7 8 9] 10 (4.7.297)
Now we highlight what we will consider as the successes (R) and failures (E):
[1 2 3 4 5 6 7 8 9] 10
[E E R E E E R E E] R
(4.7.298)

Draft
We have also 7 failures and 3 successes. In an experiment where the draws are independent (or
can be considered as independent...), the probability that we get this particular result is:
(0.8)7
(0.2)3
(4.7.299)
But the order of successes and failures in the bracketed part is irrelevant. So as we have 2
success among the 9 trial in brackets it follows that the probability of obtaining the same result
regardless of the order is then using combinatorics:
C9
2 (0.8)7
(0.2)3
=
9
2
(0.8)7
(0.2)3 ∼= 0.0603 (4.7.300)
Which corresponds to the probability of having 7 failures before the 3rd success (or otherwise
seen: 3 successes after 10 trials). This can be written with Microsoft Excel 14.0.6123 or later
(7 + 3 = 10 trials, 7 failures including 3 successes):
axiqfsxywhsƒ„@UDQDHFPDHAaHFHTHR
We now generalize the prior-previous notation by writing the number of failures k, N the total
number of trials and p the probability of success:
NB(N, k, p) = CN−1
N−k−1qk
pN−k
=
N − k − 1
N − 1
qk
pN−k
(4.7.301)
However, there are several possible notations because the previous relation is not very intuitive
to practice as may have perhaps noticed the reader. Thus, if we denote k as the number of
successes and not the number of failures, then we have (the most common writing way from
my point of view among a lot of others notations) the following probability of having N − k
success before having a number k of failures with a probability p (or of failures before having
k successes ... it’s symmetrical!):
NB(N, k, p) = Ck−1
N−1qN−k
pk
=
N − 1
k − 1
pk
qN−k
(4.7.302)
therefore the comparison with the formulation of the binomial distribution proved above is then
perhaps more obvious!
However, it is more common to write the previous relation by removing N because for the
moment the notation is still perhaps not very clear. For this, we note R the number of successes
, E the number of failures, p the probability of success and then comes the probability of having
R success after E failures (this is perhaps much more clear...):
NB(R, E, p) = CE−1
E+R−1q(E+R)−E
pE
=
E + R − 1
E − 1
pE
qR
(4.7.303)
We sometimes ﬁnd this last relation with another relation using explicitly the binomial coefﬁ-
cient:
NB(R, E, p) =
E + R − 1
E − 1
qR
pE
=
(E + R − 1)!
(E − 1)!R!
qR
pE
=
(E + R − 1)!
R!(E − 1)!
qR
pE
=
E + R − 1
R
qR
pE
(4.7.304)

Draft
The cumulative probability that we have at least R successes before the E-th failure is obviously
given by:
N
R=0
E + R − 1
E − 1
qR
pE
(4.7.305)
Remark
The name of this law comes from the fact that some statisticians use a deﬁnition of the
binomial coefﬁcient with a negative value for the expression of the function. Since this
is a rather a rare notation, we do not want lose time to prove the origin of the name.
You should also know that this law is also known as the Pascal’s law (as well as the
geometric distribution ...) in honor of Blaise Pascal and also as PPolya’s law in honor
of George Pólya.
Examples:
E1. A long-term quality control has enabled us to compute the estimator of the
proportion p of nonconforming pieces as equal to 2% at the output of a production line.
We would like to know the cumulative probability to have 200 pieces before the 3rd
defective piece appears. With Microsoft Excel 14.0.6123 or later it comes using the
negative binomial distribution:
axiqfsxywFhsƒ„@PHHDQDHFHPDIAaUUFQS7
E2. To compare with the binomial distribution, we can ask ourselves what is the cumu-
lative probability of drawing 198 non-defective parts from 201 using Microsoft Excel
14.0.6123 or later:
afsxywFhsƒ„@IWVDPHIDHFWVDIAaUTFUU7
Therefore we see that the difference is small. In fact the difference between the two laws
is in practice so small that we then use almost always the binomial law (but you should
still be careful with this choice!).
As usual, we will now determine the variance and mean of this law. Let’s start with the mean
of having R successes when the E-th failure appears knowing that the probability of a failure is
p. For this we will use a very simple and ingenious trick (all art was thinking about it...). If we
return to our initial example:
[1 2 3 4 5 6 7 8 9] 10
[R R E R R R E R R] E
(4.7.306)
and we rewrite this example as follows:
[1 2 3 4 5 6 7 8 9 10]
[R R E
=X1
R R R E
=X2
R R E
=X3
] (4.7.307)

Draft
We then notice that the third success R of the first notation can be decomposed into the sum of
three geometric random variables such that:
R = X1 + X2 + ... + Xn (4.7.308)
With in the case of this particular example n = 3 corresponding in fact to E = 3. So quite
generally the sum of n random geometric variables always gives a negative binomial distribution
if the probability p is equal for each geometric variable!!! Anyway... as we have proved the
expression of the mean and variance of the Geometric law as (thus giving us the mean rank of
the first failure):
E(X) =
1
p
V(X) =
1 − p
p2
=
q
p2
(4.7.309)
since the random variables are independent and of same parameters then it comes for the nega-
tive binomial the mean of the rank of E-th failure using the property of the mean:
E(X) = E(X1 + X2 + ... + Xn) = E(X1) + E(X2) + ...E(Xn) = nE(X) = n
1
p
= E
1
p
(4.7.310)
And therefore for the variance of the negative binomial distribution:
V(X) = V(X1 + X2 + ... + Xn) = V(X1) + V(X2) + ...V(Xn) = nV(X) = n
1 − p
p2
= E
q
p2
(4.7.311)
So the mean and variance of the rank (corresponding to the number of trials N or from another
point of view: the mean number of successes by the simple subtraction X −E) to have the E-th
failure is then to summarize:
E(X) =
E
p
V(X) = E
1 − p
p2
(4.7.312)
Thus, putting E = 1, we fall back on the mean and variance of the geometric distribution!
Now, let Y be the random variable representing the number of trials before the R-th success.
We then have the following expressions for the variance and the mean that are very common in
the literature (these expressions of mean and variance corresponds to what we can find for the
negative binomial law in Wikipedia for example):
E(Y ) = E(X − 1) = E(X) − E(1) = E(X) − 1 =
E
p
− 1 =
E − p
p
=
E(1 − p)
p
V(Y ) = V(X − 1) = V(X) + V(1) = V(X) =
E(1 − p)
p2
(4.7.313)

Draft
Example:
What is the expected number (mean) of trials we can expect before we fall on the
third non-conforming part, knowing that the probability of a non-conforming part is 2%?
E(X) =
E
p
=
3
2%
= 150 (4.7.314)
and for the standard deviation:
σ = E
q
p2
= 3
(1 − 2%)
(2%)2
= 85.732 (4.7.315)
Like always the reader will find below a plot example of the distribution and cumulative distri-
bution function for the negative binomial law of parameters NB(N, k, p) = P(N, 3, 0.6) based
on the example of the begging, but where the only difference is the probability of success where
we he have taken 60% instead of 20%.
Thus, there is 21.6% of probability of having the third success after the third successive trial (i.e.
0 trials more than the number of successes), 25.92% of probability of having the third success
after the fourth successive trial (i.e. one trial more than the number of successes), 20.7% of
probability of having the third success after the fifth successive trial (i.e. two trial more than the
number of successes) and so on...:
Figure 4.74 – Negative Binomial law NB (mass and cumulative distribution function)
The above distributions are truncated to 9 (corresponding to 12 trials) but they theoretically
continue indefinitely. What particularly distinguishes the binomial and geometric distributions
from the negative binomial are the tails of the distribution.
The binomial negative distribution has an important place in a special regression technique that
we will see in the section of Numerical Methods.

Draft
4.7.6.6 Hypergeometric Distribution
We consider to approach this function a simple example (but not very interesting in practice) that
is this of an urn containing n balls where m are black and the other m white (for several impor-
tant examples used in the industry refer to the sections of Industrial Engineering or Numerical
Methods). We take successively, and without replacement in the urn, p balls. The question is to
ﬁnd the probability that among the p balls, there is k that are black (in this statement the order
of the drawing does not interest us).
We often talk about exhaustive sampling with the hypergeometric distribution because at the
opposite of to the binomial distribution, the size of the lot which is the basis for the sampling
will appear in the law.
Remark
This is also equivalent to an non-ordered sampling without replacement (see section
Probability) with constraint on the occurrences sometimes called simultaneous sam-
pling. We will often use the hypergeometric distribution in the ﬁeld of quality and
reliability where the black balls are associated to items with defects and the white one to
items without defects.
The p balls can be chosen among n balls in Cn
p ways (thus representing the number of possible
different outcomes) with as reminder (see section Probability):
Cn
p =
n
p
=
Ap
n
p!
=
n!
p!(n − p)!
(4.7.316)
The k black balls can be chosen among the m black in Cm
k ways. The p − k white balls can be
chosen in Cn−m
p−k ways. There is therefore Cm
k Cn−m
p−k ways to have k black balls and p − k white
balls.
The searched probability is therefore given by (we will see an alternative notation in the section
of Industrial Engineering):
H(n, p, m, k) =
Cm
k Cn−m
p−k
Cn
p
=
m!
k!(m − k)!
(n − m)!
(p − k)!((n − m) − (p − k))!
n!
p!(n − p)!
(4.7.317)
and is said to follow a Hypergeometric distribution (or Hypergeometric law) and can be
obtained fortunately directly in Microsoft Excel 14.0.7153 with the function r‰€qiywFhsƒ„@A.

Draft
Examples:
E1. We want to develop a small computer program of 10, 000 lines of code (n). The
return on experience shows that the probability of failure is one bug per 1, 000 lines of
code (or 0.1% of 10, 000 lines) that corresponds to the value m.
We test about 50% of the functionalities of the software randomly before sending it to
the customer (corresponding to the equivalent of 5, 000 lines that is p). The probability
of observing 5 bugs (k) is then given with Microsoft Excel 14.0.715:
r‰€qiywhsƒ„@kDpDmDnAar‰€qiywhsƒ„@SDSHHHDI7BIHHHHDIHHHHAaPRFTP7
E2. In a small single production of a batch of 1, 000 pieces we know that 30% on average
are bad because of the complexity of the pieces and by return on experience from a
previous similiar manufacturing. We know that a customer will randomly draw 20 pieces
to decide whether to accept or reject the lot. He will not reject the lot if he finds zero
defective pieces on the 20. What is the probability of having exactly 0 defective?
ar‰€qiywhsƒ„@HYPHYQHHYIHHHAaHFHUQ7
and as we require a null draw drawing result, the calculation of the hypergeometric dis-
tribution simplifies manually to:
700
1000
699
999
698
998
697
997
...
681
981
= 0.073% (4.7.318)
It is not forbidden to make direct calculation of the mean and variance of the hypergeomet-
ric distribution, but the reader will without much trouble guess that this calculation will be ...
relatively indigestible. Then we can use an indirect method that is much more interesting!
First, the reader will perhaps, even certainly, have noticed that experienced of the hypergeomet-
ric distribution is a series of Bernoulli trials (without replacement of course!).
So we will cheat by using initially the property of linearity of the mean. We define for this
purpose a new variable corresponding implicitly in fact to the experience of the hypergeometric
distribution (a sequence of k Bernoulli trials!):
X =
k
i=1
Xi (4.7.319)
where Xi is the success of obtaining at the i-th drawing a black ball (either 0 or 1). But, we
know that for all i the random variable Xi follows a Bernoulli function for which we have
proved in our study of the Bernoulli distribution that E(Xi) = p.
Therefore, by the property of linearity of the mean we have (caution! here p is not the number
of balls, but the probability associated with an expected event!):
E(X) = E
k
i=1
Xi = kp (4.7.320)
In the Bernoulli trial, p is the probability of obtaining the desired item or event (for reminder...).
In the hypergeometric distribution what interests us is the probability of a black ball (which are

Draft
in quantity m, therefore with m white balls) compared to the total amount of n balls. And the
ratio obviously gives us this probability. Thus, we have:
µ = E(X) = kp = k
m
m + m
= k
m
n
(4.7.321)
where k is the number of trials (do not confuse with the notation of the initial statement where
it was by the variable p!). The mean gives then the average number of black balls in a drawing
of k balls among n, where m are known as being black. The reader will have noticed that the
hope of the hypergeometric distribution is the same as the binomial distribution!
To determine the variance, we use the variance of the Bernoulli distribution and the following
relation proved during the introduction of the mean and covariance at the beginning of this
chapter:
V(X + Y ) = V(X) + V(Y ) + 2cov(X, Y ) + V(X) + V(Y ) + 2(E(XY ) − E(X)E(Y ))
(4.7.322)
Recalling that we have X =
k
i=1
Xi, we get:
V(X) = V
k
i=1
Xi =
k
i=1
V(Xi) + 2
k
i=1≤1j≤n
(E(XiXj) − E(Xi)E(Xj)) (4.7.323)
However, for the Bernoulli law, we have:
V(X) = pq =
m
m + m
m
m + m
=
mm
(m + m )2
=
mm
n2
(4.7.324)
Then we ﬁrst already get:
V
k
i=1
Xi = k
mm
(m + m )2
= k
mm
n2
(4.7.325)
The calculation of E(XiXj) requires a good understanding of probabilities (this will be a good
refresh!).
The mean E(XiXj) is given (implicitly), as we know, by the weighted sum of the probabilities
that two events occur at the same time. However, our events are binary: either it is a black
ball (1) or it is a white ball (0). So all terms of the sum without two consecutive black balls
consecutively will be null!
The problem is then to calculate the probability of having two consecutive black balls and it is
thus written:
P(XiXj = 1) = P((Xi = 1) ∩ (Xj = 1)) = P(Xi = 1)PXi=1P(Xj = 1)
=
m
m + m
m − 1
(m + m) − 1
(4.7.326)
So we ﬁnally have:
E(XiXj) =
m
m + m
m − 1
(m + m) − 1
(4.7.327)

Draft
Therefore:
cov(Xi, Xj) = E(XiXj) − E(Xi)(Xj) =
m
m + m
m − 1
(m + m) − 1
−
m
m + m
2
=
−mm
(m + m)2(m + m − 1)
(4.7.328)
Finally (using the result of Gauss series seen in the section of Sequences and Series):
V(X) =
k
i=1
V(Xi) + 2
k
i=1≤1j≤n
(E(XiXj) − E(Xi)E(Xj))
= k
mm
(m + m)2
+ 2
−mm
(m + m)2(m + m − 1)
k
i=1≤1j≤n
1
= k
mm
(m + m)2
+ 2
−mm
(m + m)2(m + m − 1)
(k − 1)k
2
=
kmm (m + m − 1) − mm k(k − 1)
(m + m)2(m + m − 1)
=
kmm (m + m − 1 − k + 1)
(m + m)2(m + m − 1)
=
kmm (m + m − k)
(m + m)2(m + m − 1)
= k
m
m + m
m
m + m
m + m − k
m + m − 1
= kpq
m + m − k
m + m − 1
= kpq
n − k
n − 1
(4.7.329)
where we have used the fact that:
ij
cov(Xi, Xj) (4.7.330)
is composed of:
Ck
2 =
k
2
=
k(k − 1)
2
(4.7.331)
terms as correspond to the number of ways there are to choose a pair (i, j) with i j. Because:
V(X) = kpq
n − k
n − 1
(4.7.332)
We can write:
σ = kpq
n − k
n − 1
(4.7.333)
In the specialized literature, we often ﬁnd the variance written in the following way by noting
the expected event r and the non-expected event s:
V(X) = kpq
n − k
n − 1
= k
r
n
s
n
n − k
n − 1
=
krs(n − k)
n2(n − 1)
=
klrs
n2(n − 1)
(4.7.334)
so with l = n − k. This last notation will be very useful in the section of Numerical Methods
for our study of the Mantel-Haenszel test. Furthermore, we see that in:
σ = kpq
n − k
n − 1
(4.7.335)

Draft
there is the standard deviation as the binomial distribution, at the difference of a factor that is
noted:
fpc =
n − k
n − 1
(4.7.336)
the we found often in statistics and is named ﬁnite population correction factor.
Here is an example plot example of the distribution function and cumulative distribution for the
Hypergeometric function of parameters (n, p, m, k) = (10, 6, 5, k):
Figure 4.75 – Hypergeometric law H (mass and cumulative distribution function)
We will prove now that the Hypergeometric distribution tends to a binomial distribution since
this property is used many times in different sections of this book (especially the section of
Industrial Engineering).
To do this, we decompose:
H(n, p, m, k) =
Cm
k Cn−m
p−k
Cn
p
(4.7.337)
We then get:
Cm
k Cn−m
p−k
Cn
p
=
m!
k!(m − k)!
(n − m)!
(p − k)!((n − m) − (p − k))!
n!
p!(n − p)!
=
m!
k!(m − k)!
(n − m)!
(p − k)!((n − m) − (p − k))!
p!(n − p)!
n!
=
p!
k!(p − k)!
m
(m − k)!
(n − m)!
(p − k)!((n − m) − (p − k))!
(n − p)!
n!
= Ck
p
m
(m − k)!
(n − m)!
(p − k)!((n − m) − (p − k))!
(n − p)!
n!
(4.7.338)

Draft
For the second term:
m
(m − k)!
=
1 · 2 · 3 · ... · m
1 · 2 · 3 · ... · (m − k)
= m(m − 1)...(m − k + 1) (4.7.339)
For m → +∞ (...) all the terms are of then of the order of m. Then we have:
m(m − 1)...(m − k + 1) ∼= mk
(4.7.340)
For the third term, a identical development to the previous one provides (for sure we need that
also n → +∞ (...)):
(n − m)!
(p − k)!((n − m) − (p − k))!
∼= (n − m)p−k
(4.7.341)
And for sure... we can discuss therefore about n − m when both terms tends to inﬁnity... Ditto
for the fourth term:
(n − p)!
n!
∼= n−p
(4.7.342)
In conclusion we have:
Cm
k Cn−m
p−k
Cn
p
=
n,m→+∞
Ck
p
mk
(n − m)p−k
np
(4.7.343)
We change the notation by writing p (the number of individuals drawn) as being N. We get
then:
Cm
k Cn−m
N−k
Cn
N
=
n,m→+∞
CN
k
mk
(n − m)N−k
nN
(4.7.344)
We make another change of notation by writing b the black balls and w the white balls. We get
then:
Cb
kCn−b
N−k
Cn
N
=
n,b→+∞
CN
k
bk
(n − b)N−k
nN
= CN
k
bk
wN−k
nN
(4.7.345)
Finally, we note p the proportion of black balls and q that of white balls in the lot. We then get:
Cnp
k Cn−np
N−k
Cn
N
=
np→+∞
CN
k
(np)k
(nq)N−k
nN
= CN
k (np)k
(nq)N−k
n−N
= CN
k pk
qN−k
(4.7.346)
We ﬁnd out the binomial distribution! In practice, it is common to approximate the hyperge-
ometric distribution with a binomial distribution when the ratio of the number of individuals
from the total number of individuals is less than 10
In practice, Monte-Carlo simulations with testing adjustments (see late in this chapter) have
shown that the hypergeometric distribution could be approximated by a Normal distribution
(very important case in contingency statistical tests that we will study in the section of Theoret-
ical computing) if the following three conditions are met simultaneously:
k 9
n − m
m
k 9
m
n − m
k
n
10
(4.7.347)

Draft
Thus graphically and approximately...:
Figure 4.76 – Conditions of application of the approximation by a Normal distribution
Thus:
H(k, m, n) ∼= N(µ, σ) = N k
m
n
,
krs(n − k)
n2(n − 1)
= N k
m
n
,
kmm (n − k)
n2(n − 1)
(4.7.348)
4.7.6.7 Multinomial Distribution
The multinomial distribution (also called because it involves several times the binomial coefﬁ-
cient) is a law applicable to n distinguishable events, each with a given probability, which occur
one or more times and it is not necessarily ordered. This is a frequent case in marketing re-
search and that will be useful to build the statistical McNemar test that we will study much later
(see section Theoretical Computing). We also use this law in quantitative ﬁnance (see section
Economy).
More technically, consider the space of events Ω = {1, 2, ..., m} with the probabilities P({i}) =
pi, i = 1, 2, ..., n. We take n times with a given element of Ω with replacement (see chapter
Probability) with the probability pi, i = 1, 2, ..., n. We will search what is the probability of
such a non-necessarily ordered the event 1, k1 times, event 2, k2 times and this on a sequence
of n drawings.
Remark
This is equivalent to the study of a sampling with replacement (see section Probability)
and constraints on the occurrences. So without constraints we will see with an example
that we fall back on a sampling with simple replacement.

Draft
We saw in the section of Probabilities, that if we take a set of events with multiple outcomes,
then different combinations of sequences we can get taking p selected elements among n is
given by:
Cn
p =
n!
p!(n − p)!
(4.7.349)
We have therefore:
Cn
k1
=
n!
k1!(n − k1)!
(4.7.350)
different ways to get k1 times a given event. Thus an associated probability of:
P(n, k1) = Cn
k1
pk1
1 qn−k1
1 = Cn
k1
pk1
1 (1 − p1)n−k1
(4.7.351)
Now comes the particularity of the multinomial distribution!: there are no failures in contrast to
the binomial distribution. Each pseudo-failure can be considered as a subset draw of k2 items
from the n − k1 remaining elements. Thus the term:
Cn
k1
pk1
1 qn−k1
1 (4.7.352)
will be written on the whole experience if we consider a particular case limited to two types of
events:
Cn
k1
pk1
1 qn−k1
1 := Cn
k1
pk1
1 (Cn−k1
k2
pk2
2 ) (4.7.353)
so with:
Cn−k1
k2
=
(n − k1)!
k2!((n − k1) − k2)!
(4.7.354)
which gives us the number of different times to get k2 times a second event because in the whole
sequence of n elements k1 of them have already been taken so have now only n − k1 remaining
on which we can get the k2 desired.
These relations then show us that this is a situation where each event probability is considered
as a binomial (hence its name ...).
So we have in the case of two sets of t-uples:
Cn
k1
pk1
1 Cn−k1
k2
pk2
2 = Cn
k1
Cn−k1
k2
pk1
1 pk2
2 =
n!
k1!(n − k1)!
(n − k1)!
k2!((n − k1) − k2)!
pk1
1 pk2
2
=
n!
k1!k2!
1
n − k1 − k2
pk1
1 pk2
2
(4.7.355)
and because:
−k1 − k2 = −n (4.7.356)
we get:
P =
n!
k1!k2!
1
0!
pk1
1 pk2
2 =
n!
k1!k2!
pk1
1 pk2
2 (4.7.357)

Draft
and we see that the construction of this distribution therefore requires that:
i
pi = 1
i
ki = n (4.7.358)
Thus, by induction we have the probability M we were looking for and called the Multinomial
function and using previous relation given by:
M =
n!
m
i=1
ki!
m
i=1
pki
i =
m
i=1
ki !
m
i=1
ki!
m
i=1
pki
i (4.7.359)
in the spreadsheet software Microsoft Excel 11.8346, the term:
m
i=1
ki !
m
i=1
ki!
(4.7.360)
is named multinomial coefﬁcient and is available under the name of the function
w…v„sxywsev@ A. In the literature we also ﬁnd this term sometimes under the following re-
spective notations:
k1 + k2 + ... + km
k1, k2, ..., km
m
k1, k2, ..., km
(4.7.361)
Theorem 4.27. We will show now that the multinomial distribution is effectively a probability
distribution (because we could doubt ...). If this is the case, as we know it, the sum of the
probabilities must be equal to 1.
Proof 4.27.1. Recall that in the chapter of Calculus we proved that (binomial theorem):
(x + y)n
=
n
k=0
Cn
k xk
yn−k
(4.7.362)
Now do a little bit of notation:
(x1 + x2)n
=
n
k1=0
Cn
k xk1
1 xn−k1
2 =
n
k1=0
n!
k1!(n − k1)!
xk1
1 xn−k1
2 (4.7.363)
and this time a change of variables:
(x1 + x2)n
=
n
k1=0
Cn
k xk1
1 xn−k1
2 =
n
k1=0
n!
k1!k2!
xk1
1 xk2
2 (4.7.364)
This last relation (which is a special case of the two terms multinomial theorem) will be useful
to us to show that the multinomial distribution is effectively a probability distribution. We also
take the special case with two groups of drawing:
M =
n!
m
i=1
ki!
m
i=1
pki
i =
n!
k1!k2!
pk1
1 pk2
2 (4.7.365)

Draft
which can is also written by the construction of the multinomial distribution:
M =
n!
k1!(n − k1)!
pk1
1 pn−k1
2 (4.7.366)
and therefore, the sum must be equal to the unit such that:
n
k1=0
n!
k1!(n − k1)!
pk1
1 pn−k1
2 = 1 (4.7.367)
To check this we use the multinomial theorem shown above:
(p1 + p2)n
=
n
k1=0
n!
k1!k2!
pk1
1 pk2
2 (4.7.368)
However, by construction of the multinomial sum of probabilities is unitary, we have effectively:
(p1 + p2)n
= (1)n
= 1 =
n
k1=0
n!
k1!(n − k1)!
pk1
1 pn−k1
2 (4.7.369)
Q.E.D.
Examples:
E1. We launch an unbiased die 12 times. What is the probability that all 6 faces
appear the same number of times (not necessarily consecutively!) that means twice for
each:
M =
n!
m
i=1
ki!
m
i=1
pki
i =
12!
6
i=1
2!
6
i=1
1
6
2
=
12!
26612
= 0.34% (4.7.370)
where we see well that m is the number of success groups.
E2. We launch an unbiased die 12 times. What is the probability that a single unique face
appears 12 times (hence the 1 appears 12 times, or the 2 or the 3, etc.):
M =
n!
m
i=1
ki!
m
i=1
pki
i =
12!
1
i=1
12!
1
i=1
1
6
12
=
12!
612
(4.7.371)
So we end up with this last example known a being a binomial distribution result.

Draft
4.7.6.8 Poisson Distribution
For some rare events, the probability p is very small and tends to zero. However, the average
value np tends to a fixed value as n tends to infinity.
We start from a binomial distribution with mean µ = np that we will assume finite when n
tends to infinity.
The probability of k successes after n trials is (Binomial distribution):
B(n, k) = Cn
k pk
(1 − p)n−k
= Cn
k pk
qn−k
=
n
k
pk
qn−k
(4.7.372)
By writing p =
m
n
(where m will be temporarily the new notation for the mean according to
µ = np), this expression can be rewritten as:
B(n, k) =
n
k
pk
qn−k
=
n!
k!(n − k)!
n
m
k
1 −
m
n
n−k
(4.7.373)
By grouping the terms, we can put the value under the form:
B(n, k) =
mk
k!
1 −
m
n
n n!
(n − k)!nk 1 −
m
n
k
(4.7.374)
We recognize that when n tends to infinity, the second factor of the product has for limit e−µ
(see Functional Analysis). The third factor, since we are interested to the small values of k (the
probability of success is very small), its limit for n tending to infinity is equal to 1.
This technique of passing to the limit is sometimes named in this context: the Poisson limit
theorem.
So we get the Poissons distribution (or Poisson law), also sometimes named the law of rare
events therefore given by:
P(µ, k) =
µk
k!
e−µ
(4.7.375)
which can be obtained in Microsoft Excel 11.8346 with the function €ysƒƒyx@ A and in prac-
tice and the specialized literature is often indicated by the letter u.
It is indeed a probability distribution since using the Taylor series (see chapter Sequences And
Series), we show that the sum of the cumulative probabilities is:
+∞
k=0
µk
k!
e−µ
= e−µ
+∞
k=0
µk
k!
= e−µ
eµ
= 1 (4.7.376)

Draft
Remark
We will frequently encounter this distribution in different sections of the book such as
in the study of preventive maintenance in the section of Industrial Engineering or in the
section of Quantitative Management for the study of queuing theory (the reader can refer
to them for interesting and pragmatic examples) and ﬁnally in the ﬁeld of life and non-life
insurance.
Here is a plot example of the Poisson distribution and cumulative distribution function with
parameter µ = 3:
Figure 4.77 – Poisson law P (mass and cumulative distribution function)
This distribution is important because it describes many processes whose probability is small
and constant. It is often used in the queing theory (waiting time), acceptability and reliabil-
ity test and statistical quality control. Among other things, it applies to processes such as the
emission of light quanta by excited atoms, the number of red blood cells seen under the micro-
scope, the number of incoming calls to a call center. The Poisson distribution is valid for many
observations in nuclear and particle physics.
The mean (average) of the Poisson distributions is (we use the Taylor series of the exponential):
µ = E(k) =
+∞
k=0
kP(µ, k) =
+∞
k=0
k
µk
k!
e−µ
= e−µ
µ
+∞
k=1
µk−1
(k − 1)!
= e−µ
µeµ
= µ (4.7.377)
and gives the average number of times that you get the desired outcome. This result may seem
confusing .... the mean is expressed by the mean?? Yes must simply not forget that it is given
since the beginning by:
µ = np (4.7.378)
Remark
For more details the reader may also refer to the subsection on estimators later in this
section.

Draft
The variance of the Poisson distribution function is itself given by (again we use the Taylor
series):
σ2
= V(k) =
+∞
k=0
[k − µ2
]Pk =
+∞
k=0
[k − µ2
]
µk
k!
e−µ
=
+∞
k=0
k2 µk
k!
e−µ
−
+∞
k=0
2kµ
µk
k!
e−µ
+
+∞
k=0
µ2 µk
k!
e−µ
=
+∞
k=0
k2 µk
k!
e−µ
− 2µµ
+∞
k=1
kµk−1
k(k − 1)!
e−µ
+ µ2
+∞
k=0
µk
k!
e−µ
=
+∞
k=0
k2 µk
k!
e−µ
− 2µ2
+∞
k=1
µk−1
(k − 1)!
e−µ
+ µ2
· 1 =
+∞
k=0
k2 µk
k!
e−µ
− 2µ2
+ µ2
= e−µ
+∞
k=0
k2 µk
k!
− µ2
= µe−µ
+∞
k=1
k
µk−1
(k − 1)!
− µ2
=
s=k−1
µe−µ
+∞
s=0
(s + 1)
µs
s!
− µ2
= µe−µ
+∞
s=0
s
µ
s!
+
+∞
s=0
µs
s!
− µ2
= µe−µ
µ
+∞
s=1
µs−1
(s − 1)!
+ eµ
− µ2
= µe−µ
(µeµ
+ eµ
) − µ2
= µe−µ
µeµ
+ µe−µ
eµ
− µ2
= µ2
− µ − µ2
= µ
(4.7.379)
always with:
µ = np (4.7.380)
The important fact for the Poisson distribution is that the variance that is equal to the mean is
name the equidispersion property of the Poisson distribution. This is a property often used
in practice as an indicator to identify whether the data (with discrete support) are distributed
according to a Poisson distribution.
The theoretical laws of statistical distributions are determined assuming completion of an infi-
nite number of measurements. It is obvious that we can only perform in practice a finite number
N. Hence the utility to establish correspondence between the theoretical and experimental val-
ues. For the experimental values we obviously obtain only an approximation whose validity is,
however, often accepted as sufficient.
Now we will prove an important property of the Poisson distribution in the field of engineering
and statistics that we name stability by addition. The idea is as follows:
Let X and Y be two independent random variables with Poisson distribution of respective
parameters λ and µ. We want to ensure that their sum is also a Poisson distribution:
X + Y = λ + µ, (4.7.381)
See this:
P(X + Y = k) =
k
i=0
P [(X = i) ∩ (Y = k − i)] =
k
i=0
P(X = i)P(Y = k − i) (4.7.382)

Draft
because the events are independent. Then we have:
P(X + Y = k) =
k
i=0
P(X = i)P(Y = k − i) =
k
i=0
λi
e−λ
i!
µk−i
eµ
(k − i)!
=
e−(λ+µ)
k!
k
i=0
k!
i!(k − i)!
λi
µk−i
(4.7.383)
However, by applying the binomial theorem (see section Calculus):
k
i=0
k!
i!(k − i)!
λi
µk−i
=
k
i=0
Ck
i λi
µk−i
= (λ + µ)k
(4.7.384)
So in the end:
P(X + Y = k) = (λ + µ)k e−(λ+µ)
k!
(4.7.385)
and therefore the Poisson distribution is stable by addition. So any Poisson distribution where
the parameter is verbatim indefinitely dividable into a finite or infinite sum of independent
Poisson distributions.
4.7.6.9 Normal Gauss-Laplace Distribution
This characteristic is the most important function of distribution in the field of statistics follow-
ing a famous theorem named the central limit theorem, which as we will see later, permits to
prove (among other things) that any sum of independent identically distributed random variables
with a finite mean and variance converges to a Laplace-Gaussian function (Normal distribution).
It is therefore very important to focus your attention on the developments that will be presented
right now!
Let start from a binomial function and make tender the number of trials n to infinity. If p is set
from the beginning , the mean µ = np also tends to infinity, furthermore the standard deviation
σ = npq also tends to infinity.
Remark
The case where p varies and tends to 0 while keeping fixed the mean has already been
studied during the study of the Poisson function.
If we want to calculate the limit of the binomial function, it will then be necessary to make a
change of origin, which stabilizes the mean, to 0 for example, and a change of unit change that
stabilizes the standard deviation to 1 for example.
Let us now denote by Pn(k) the binomial probability of k success and let’s see first how Pn(k)

Draft
vary with k and calculate the difference:
Pn(k + 1) − Pn(k) =
n
k + 1
pk+1
qn−k−1
−
n
k
pk
qn−k
=
n
k
pk
qn−k (n − k)p
(k + 1)q
− 1 = Pn(k)
np − k − q
(k + 1)q
(4.7.386)
We conclude that Pn(k) is an increasing function of k, as np − k − q is positive (for n, p and
q fixed). Too see it, juste take a few values (of the right term of the equality) or to observe the
graph of the binomial distribution function, remembering that:
µ = np (4.7.387)
As q 1 it is therefore evident that the value of k close to the mean µ = np of the binomial
distribution is the maxima of Pn(k).
On the other hand the difference Pn(k + 1) − Pn(k) is the increase rate of the function Pn(k).
We can then write:
∆Pn(k)
∆k
=
Pn(k + 1) − Pn(k)
(k + 1) − k
(4.7.388)
as being the slope of the function.
We now define a new random variable such that its average is zero (negligible variations) and
its standard deviation equal to the unit (this will be a centered-reduced variable in other words).
Then we have:
x =
k − np
√
npq
(4.7.389)
Then we also have with this new random variable:
∆x =
(k + 1) − np
√
npq
−
k − np
√
npq
=
[(k + 1) − np] − (k − np)
√
npq
(4.7.390)
Let us write denote F(x) as being Pn(k) calculated using the new random variable with zero
mean and unit standard deviation which we seek the expression when n tends to infinity.
Let go back to:
∆Pn(k)
∆k
=
np − k − q
(k + 1)q
Pn(k) =
−(k − np) − q
(k + 1)q
Pn(k) (4.7.391)
To simplify the study of this relation when n tends to infinity and k to mean µ = np, multiply
both sides by npq/
√
npq:
∆Pn(k)
∆k
npq
√
npq
=
np − k − q
(k + 1)q
npq
√
npq
Pn(k) =
−(k − np) − q
(k + 1)q
npq
√
npq
Pn(k) (4.7.392)
We rewrite now the right-hand side of this equality. Then we get:
−(k − np) − q
(k + 1)
np
√
npq
Pn(k) =
[−(k − np) − q]np
(k + 1)
√
npq
Pn(k) (4.7.393)

Draft
And now let us rewrite the left term of the prior-previous relation. We then get:
∆Pn(k)
∆k
npq
√
npq
=
∆Pn(k)
(k + 1) − 1
npq
√
npq
=
∆Pn(k)
[(k + 1) − np] − (k − np)
npq
√
npq
=
∆Pn(k)
[(k + 1) − np] − (k − np)
√
npq
npq
√
npq
√
npq
=
∆Pn(k)
[(k + 1) − np] − (k − np)
√
npq
=
∆Pn(k)
∆x
(4.7.394)
After passing to the limit for n tending to infinity we have in a first time for the denominator of
the second term of the prior-previous relation:
[−(k − np) − q]np
(k + 1)
√
npq
Pn(k) (4.7.395)
the following simplification:
(k + 1)
√
npq =
n→+∞
k
√
npq (4.7.396)
Thus:
−[(k − np) + q]np
k
√
npq
Pn(k) = −
k − np
k
√
npq
npPn(k) −
qnp
k
√
npq
Pn(k) (4.7.397)
and in a second time, taking into account that the considered values of k are then in the neigh-
borhood of the mean np, we get:
k − np
k
√
npq
np =
np
k
k − np
√
npq
∼=
n→+∞
k − np
√
npq
= x (4.7.398)
and:
qnp
k
√
npq
∼=
n→+∞
qnp
np
√
npq
=
q
√
npq
∼=
n→+∞
0 (4.7.399)
Thus:
−[(k − np) + q]np
k
√
npq
∼=
n→+∞
−x (4.7.400)
and as:
Pn(k) :=
n→+∞
F(x) (4.7.401)
where F(x) represents (awkwardly) for the few lines that follow, the density function as n tends
to infinity.
Finally we have:
dF(x)
dx
= −xF(x) (4.7.402)

Draft
This relation can also be rewritten rearranging the terms:
1
F(x)
dF(x) = −xdx (4.7.403)
and by integrating both sides of this equality we obtain (see section Differential And Integral
Calculus):
ln(F(x)) = −
x2
2
+ cte
(4.7.404)
The following function is a solution of the above equation:
F(x) = λe
−
x2
2 (4.7.405)
Effectively:
ln


λe
−
x2
2


 = ln(λ) + ln


e
−
x2
2


 = cte
+ −
x2
2
= −
x2
2
+ cte
(4.7.406)
The constant is determined by the condition that:
+∞ˆ
−∞
F(x)dx (4.7.407)
which represents the sum of all probabilities, that mus be equal to 1. We prove for this that we
need to have:
λ =
1
√
2π
(4.7.408)
Proof 4.27.2. We have:
+∞ˆ
−∞
e
−x2
2 dx =
+∞ˆ
−∞
e
−
x
√
2
2
dx =
√
2
+∞ˆ
−∞
e−z2
dz (4.7.409)
So let us focus on the last term of this equality. Thus:
I =
+∞ˆ
−∞
e−x2
dx = 2
+∞ˆ
0
e−x2
dx (4.7.410)
since e−x2
is an even function (see section Functional Analysis). Now we write the square of
the integral as follows:
I2
= 4 lim
R→+∞




Rˆ
0
e−x2
dx




Rˆ
0
e−y2
dy



 = 4 lim
R→+∞


Rˆ
0
Rˆ
0
e−(x2+y2)
dxdy

 (4.7.411)

Draft
and make a change of variable passing in polar coordinates, therefore we also use the Jacobian
in these same coordinates (see section Differential And Integral Calculus):
I2
= 4 lim
R→+∞
ˆ ˆ
e−r2
rdrdφ = 4 lim
R→+∞






π
2ˆ
0
Rˆ
0
e−r2
rdrdφ






= 4
π
2
lim
R→+∞


Rˆ
0
e−r2
rdr

 = 2π −
1
2
e−r2
+∞
0
= 2π 0 +
1
2
= π
(4.7.412)
Therefore:
I =
√
π (4.7.413)
By extension for e
−
x2
2 we have:
I = λ−1
=
√
2π (4.7.414)
Q.E.D.
We thus obtain the standard Normal distribution noted as probability density function (noted
with the capital letter F that can unfortunately lead to confusion in the present development
with the notation of the cumulative distribution function... we apologize...):
F(x) =
1
√
2π
e
−
x2
2 (4.7.415)
which can be calculated in Microsoft Excel 11.8346 with the function xy‚wƒhsƒ„@ A.
For information, a variable following a Normal centered reduced distribution is by tradition
often noted Z (Zentriert in German).
Returning to non-normalized variables:
x =
k − np
√
npq
=
k − µ
σ
(4.7.416)
so we get the Gauss-Laplace function (or Gauss-Laplace law) or also named Normal dis-
tribution given in the form of probability density in this book by:
P(k, µ, σ) = N(µ, σ2
) =
1
σ
√
2π
e
−
(k − µ)2
2σ2 (4.7.417)
The cumulative probability (distribution cumulative function) to have a certain value k is obvi-
ously given by:
P(k ≤ x) = Φ(x) =
1
σ
√
2π
xˆ
−∞
e
−
(k − µ)2
2σ2
dk (4.7.418)

Draft
Here is a plot example of the distribution and cumulative distribution function for the Normal
law with the parameters example (µ, σ) = (0, 1) that is therefore the standard centered reduced
Normal distribution:
Figure 4.78 – Normal law N (mass and cumulative distribution function)
This law governs under very general conditions, and often encountered, in many random phe-
nomena. It is also symmetrical with respect to the mean (this is important to remember!).
We will now show that µ well represents the mean (or average) of x (this is a bit silly but we
can still check ...):
E(X) =
+∞ˆ
−∞
xf(x)dx =
1
σ
√
2π
+∞ˆ
−∞
e
−
(x − µ)2
2σ2
dx (4.7.419)
We put:
u =
x − µ
σ
(4.7.420)
We then have:
E(X) =
+∞ˆ
−∞
xf(x)dx =
1
σ
√
2π
+∞ˆ
−∞
(σu + µ)e
−
u2
2 σdu
= σ
1
σ
√
2π
+∞ˆ
−∞
σue
−
u2
2 du + σ
1
σ
√
2π
+∞ˆ
−∞
µe
−
u2
2 du
= σ
1
√
2π
+∞ˆ
−∞
ue
−
u2
2 du +
1
√
2π
µ
+∞ˆ
−∞
e
−
u2
2 du
(4.7.421)

Draft
Let us now calculate the first integral:
J =
+∞ˆ
−∞
ue
−
u2
2 du = −e
−
u2
2
+∞
−∞
= e
−
u2
2
−∞
+∞
= 0 − 0 = 0 (4.7.422)
So we finally get:
E(X) =
1
√
2π
µ
+∞ˆ
−∞
e
−
u2
2 du =
1
√
2π
µ
√
2π = µ (4.7.423)
Remarks
R1. The reader might find confusing at first that the parameter of a function is one of the
results that we seek of this same function (as for the Poisson distribution). What bothers
is to put in practice such a thing. In fact, everything will be more clear when we will
discuss later in this chapter the concepts of likelihood estimators.
R2. It could be interesting to know for the reader that in practice (finance, quality as-
surance, etc.) it is common to have to calculate only mean only positive values of the
random variable which is then naturally defined as positive mean and given by:
E+
(X) =
1
σ
√
2π
+∞ˆ
0
xe
−
1
2
x − µ
σ
2
dx (4.7.424)
We will see a practical example of this last relation in the section Economy during our
study of the theoretical model of speculation of Louis Bachelier.
Also we will prove now (...) that σ is the standard deviation of X (in other words to prove that
V(X) = σ2
) and for this we recall that we had prove that (Huyghens relation):
V(X) = E(X2
) − E(X)2
(4.7.425)
We already know that at the level of the notations we have:
E(X) = µ ⇒ E(X)2
= µ2
(4.7.426)
then we first calculate E(X2
):
E(X2
) =
ˆ
x2
f(x)dx =
1
σ
√
2π
+∞ˆ
−∞
x2
e
−
1
2
x − µ
σ
2
dx (4.7.427)
Let y = (x − µ)/
√
2σ that therefore leads us to:
E(X2
) =
ˆ
x2
f(x)dx =
1
σ
√
π
+∞ˆ
−∞
(y
√
2σ + µ)2
e−y2
dx
=
2
σ
√
π
+∞ˆ
−∞
y2
e−y2
dy + 2µσ
2
π
+∞ˆ
−∞
ye−y2
dy +
µ2
√
π
+∞ˆ
−∞
e−y2
dy
(4.7.428)

Draft
And we know that (already proved above):
+∞ˆ
−∞
ye−y2
dy = 0
+∞ˆ
−∞
e−y2
dy =
√
π (4.7.429)
It remains to calculate therefore only the first integral. To do this, we proceed by integration by
parts (see Differential and Integral Calculus):
bˆ
a
f(t)g (t)dt = f(t)g(t)|b
a −
bˆ
a
f (t)g(t)dt (4.7.430)
leads us to:
+∞ˆ
−∞
y2
e−y2
dy =
+∞ˆ
−∞
y ye−y2
dy = −y
e
−
y2
2
2
b
a
+
+∞ˆ
−∞
e−y2
2
dy =
√
π
2
(4.7.431)
Then we get:
E(X2
) =
2
√
π
σ2
√
π
2
+ 2µσ
2
π
0 +
µ2
√
pi
pi = µ2
+ σ2
(4.7.432)
And finally:
E(X) = E(X2
) − E(X)2
= (µ2
+ σ2
) − µ2
= σ2
(4.7.433)
An additional signification of the standard deviation of Gauss-Laplace distribution is a measure
of the width of the distribution as (this can be checked only with the aid of integration by using
numerical methods) for any non-zero mean and standard deviation we have (thanks to John
Cannin for the LATEXfigure):
68.2%
95%
99.7%
34.1% 34.1% 13.6%13.6% 2.1%2.1%
−3 −2 −1 0 1 2 3
Standard deviations
Frequency
Figure 4.79 – Sigma intervals for the Normal distribution
An additional signification of the standard deviation of Gauss-Laplace distribution is a measure
of the width of the distribution as (this can be checked only with the aid of integration by using
numerical methods) for any non-zero mean and standard deviation we have:

Draft
The width of the interval has a great importance in the interpretation of uncertainties measure-
ment. The presentation of a result like ¯N ± σ has for signification that the average value has
about 68.3% chance (probability) to lie between the limits of ¯N − σ and ¯N + σ or has approxi-
mately 95.4% to lie between the limits of ¯N − 2σ and ¯N + 2σ etc.
Remarks
This concept is widely used in quality management in industrial business especially with
the Six Sigma methodology (see Industrial Engineering) which requires a mastery of 6
around each side of the mean (!) of the manufacturing (or anything else whose deviation
is measured).
The second column of the table can easily be obtained with Maple 4.00b (or also with
the spreadsheet software from Microsoft). For example for the first line:
bƒXaev—lf@int@IGsqrt@PB€iABexp@Ex” PGPADxaEIFFIAAY
and the first row of the third column:
b@IEƒABIiTY
If the Normal distribution was not centered, then we just would write for the second
column:
bƒXaev—lf@int@IGsqrt@PB€iABexp@E@xEmuA” PGPADxaEIFFIAAY
and so on for any deviation and mean we will then obtain exactly the same intervals!!!
The Gauss-Laplace distribution is also not only a tool for data analysis but also for data gener-
ation. Indeed, this distribution is one of the largest used in the world of multinationals that use
statistical tools for risk management, project management and simulation where a large number
of random variables are to be controled. The best examples of applications use the softwares
CrystalBall or Palisade @Risk (this last one being my favorite...).
In this context of application (project management), it is also very common to use the sum (task
duration) or the product of random variables (customer uncertainty) following Gauss-Laplace
distributions. We will see now how to to calculate this:
4.7.6.9.1 Sum of two random Normal variables
Let X, Y be two independent random variables. Suppose that X follows the distribu-
tion N(µ1, σ2
1) and that Y follows the distribution N(µ2, σ2
2). Then the random variable
Z = X + Y has a density equal to the convolution product of fx and fy. That is to say:
fz(s) =
+∞ˆ
−∞
fX(x)fY (y)(s − x)dx =
1
2πσ1σ2
+∞ˆ
−∞
e
−
(x − µ1)2
2σ2
1 e
−
(s − x − µ2)2
2σ2
2 dx (4.7.434)

Draft
which is equivalent to the joint product (see Probabilities) of the probabilities of occurrence of
the two continuous variables (remember the same kind of calculation in discrete form!).
To simplify the expression, make the change of variable t = x − µ1 and let us write a =
µ1 + µ2 − s, σ = σ2
1 + σ2
2.
As:
s − y − µ2
2
2
= − −s + y + µ2
2
2
= −s + y + µ2
2
2
= (t + a)2
(4.7.435)
we get after a hard to guess rearrangement trick:
fz(s) =
1
2πσ1σ2
+∞ˆ
−∞
e
−
t2
2σ2
1 e
−
(t + a)2
2σ2
2 dt =
1
2πσ1σ2
+∞ˆ
−∞
e
−
σt +
aσ2
1
σ
2
+
σ2
1a2
σ2
2
σ2
2σ2
1σ2
2 dt
=
1
2πσ1σ2
e
−
a2
2σ2
+∞ˆ
−∞
e
−
σt +
aσ2
1
σ
2
2σ2
1σ2
2 dt
(4.7.436)
We write:
u =
σt +
aσ2
1
σ√
2σ1σ2
⇒
du
dt
=
σ
√
2σ1σ2
⇒ dt =
√
2σ1σ2
du
σ
(4.7.437)
Then:
fz(s) =
1
2πσ1σ2
e
−
a2
2σ2
+∞ˆ
−∞
e
−
σt +
aσ2
1
σ
2
2σ2
1σ2
2 dt =
1
√
2πσ
e
−
a2
2σ2
+∞ˆ
−∞
e−u2
du
(4.7.438)
Knowing that (proved above):
+∞ˆ
−∞
e−u2
du =
√
π (4.7.439)
and:
a2
= (−s + µ1 + µ2)2
= (s − µ1 − µ2)2
(4.7.440)
our relation becomes:
fz(s) =
1
σ
√
2
√
π
e
−
(s − µ1 − µ2)2
2σ2 (4.7.441)

Draft
We recognize the expression of the Gauss-Laplace distribution (Normal law) of mean µ1 + µ2
and standard deviation σ = σ2
1 + σ2
2. Therefore, X + Y follows the distribution as written by
the physicist (both argument have same units):
N µ1 + µ2, σ2
1 + σ2
2 (4.7.442)
and as noted by most mathematicians, statisticians:
N µ1 + µ2, σ2
1 + σ2
2 (4.7.443)
The fact that the sum of two Normal distributions always give also a Normal distribution is what
we name in statistics stability of the sum of the Gauss-Laplace distribution (Normal law). We
will find such properties for other distribution that will be discussed later.
So as well as for the Poisson distribution, any Normal distribution whose parameters are known
is verbatim indefinitely divisible into a finite or infinite number of independent Normal distri-
bution that are summed as:
N(µ, σ2
) =
n
i=1
N
µ
n
,
σ2
n
(4.7.444)
Remark
The families of stable distribution by the sum is an important field of study in physics, fi-
nance and statistics called Levy alpha-stable distribution. If time permits, I will present
the details of this extremely important study in this chapter.
4.7.6.9.2 Product of two random Normal variables
Let X, Y be two real independent random variables. We denote by fX and fY the cor-
responding densities and we seek to determine the density of the variable Z = XY (very
important case, particularly in engineering).
Let F denote the density function of the pair (X, Y ). Since X, Y are independent (see section
Probabilities):
f(x, y) = fX(x)fY (y) (4.7.445)
The distribution function of Z is:
F(Z) = P(Z ≤ z) = P(XY ≤ z) =
¨
D
f(x, y)dxdy =
¨
D
fX(x)fY (y)dxdy (4.7.446)
where D = {(x, y)|xy z}.
D can be rewritten as a disjoint union (we do this for anticipating in the future change of
variables a division by zero):
D = D1 D2 D3 (4.7.447)

Draft
with:
D1 = (x, y) ∈ R2
|xy ≤ z ∧ x 0
D2 = (x, y) ∈ R2
|xy ≤ z ∧ x 0
D3 = (x, y) ∈ R2
|xy ≤ z ∧ x = 0
(4.7.448)
We have:
F(Z) =
¨
D1
fX(x)fY (y)dxdy +
¨
D2
fX(x)fY (y)dxdy +
¨
D3
fX(x)fY (y)dxdy
=0
(4.7.449)
The last integral is equal to zero because D3 is of measurement (thickness) zero for the integral
along x.
We then perform the following change of variable:
x = xt = xy (4.7.450)
The Jacobian of the transformation (see section Differential and Differential Calculus) is:
1 0
−t/x2
1/x
=
1
|x|
(4.7.451)
Thus:
F(z) =
zˆ
−∞
+∞ˆ
0
fX(x)fY (t/x)
|x|
dxdt +
zˆ
−∞
0ˆ
−∞
fX(x)fY (t/x)
|x|
dxdt =
zˆ
−∞
+∞ˆ
−∞
fX(x)fY (t/x)
|x|
dxdt
(4.7.452)
Let fZ be the density of the variable Z. By deﬁnition:
F(Z ≤ z) = F(z) =
zˆ
−∞
fZ(t)dt (4.7.453)
On the other hand:
F(z) =
zˆ
−∞
+∞ˆ
−∞
fX(x)fY (t/x)
|x|
dxdt (4.7.454)
as we have seen. Therefore:
fZ(t) =
+∞ˆ
−∞
fX(x)fY (t/x)
|x|
dx (4.7.455)
What is a bit sad is that in the case of a Gauss-Laplace distribution (Normal distribution), this
integral can only be easily calculated numerically ... it is then necessary to use Monte-Carlo
integration type methods (see section Theoretical Computing).
However according to some research done on the Internet, but without certainty, this integral
may be calculated and give a new distribution called Bessel distribution.

Draft
4.7.6.9.3 Bivariate Normal Distribution
If two Normal distributed random variables are independent, we know that their joint
probability is equal to the product of their probabilities. So we have:
P = P1P2 =
1
√
2πσ1
e
−
(x − µ1)2
2σ2
1
1
√
2πσ2
e
−
(x − µ2)2
2σ2
2 =
1
2πσ1σ2
e
−
(x − µ1)2
2σ2
1
−
(x − µ2)2
2σ2
2
(4.7.456)
Now comes an approach that we will often find in the follow developments: to generalize
simple algebra models, you have to think in a Linear Algebra way! Therefore we are left with
two vectors involving a scalar product:
P = P1P2 =
1
2πσ1σ2
e
−
(x − µ1)2
2σ2
1
−
(x − µ2)2
2σ2
2 =
1
2πσ1σ2
e
−
1
2


x1 − µ1
x2 − µ2


T






x1 − µ1
2σ2
1
x2 − µ2
2σ122






(4.7.457)
But we can do even better because for the moment there is no added value to this notation!
Effectively a subtle idea is to involve the determinant of a matrix (see section Linear Algebra)
and the inverse of this same matrix in the previous relation:
P = P1P2 =
1
2πσ1σ2
e
−
1
2


x1 − µ1
x2 − µ2


T






x1 − µ1
2σ2
1
x2 − µ2
2σ122






=
1
2π(σ2
1σ2
2 − 0 · 0)1/2
e
−
1
2


x1 − µ1
x2 − µ2


T







1
σ2
1
0
0
1
σ2
2









x1 − µ1
x2 − µ2


=
1
2π
σ2
1 0
0 σ2
2
1/2
e
−
1
2


x1 − µ1
x2 − µ2


T 

σ2
1 0
0 σ2
2


−1

x1 − µ1
x2 − µ2


(4.7.458)
We thus find a particular case of the variance-covariance matrix. In the field of the bivariate
Normal distribution is it is customary to write this last relation in the following form:
f(X1, X2) =
1
2π|Σ|1/2
e
−
1
2
−−−−→
(x−µ)T Σ−1
−−−−→
(x−µ)
(4.7.459)
If we make a plot of this function we get:

Draft
Figure 4.80 – Plot of the bivariate Normal function in MATLAB
or another one (not with the same values) with corresponding projections:
Figure 4.81 – Plot of the bivariate Normal function with pgfplots
Now consider the important case in engineering, astronomy and quantum physics by returning

Draft
to the following notation:
P = f(X1, X2) =
1
2πσ1σ2
e
−
(x − µ1)2
2σ2
1
−
(x − µ2)2
2σ2
2 (4.7.460)
and by focusing on to the iso-lines such that for any pair of values of the two random variables,
we have:
1
2πσ1σ2
e
−
(x − µ1)2
2σ2
1
−
(x − µ2)2
2σ2
2 = cte
(4.7.461)
By doing some very basic algebraic manipulations, we get:
−
(x − µ1)2
2σ2
1
−
(x − µ2)2
2σ2
2
= ln 2πσ1σ2cte
(4.7.462)
Thus:
(x − µ1)2
2σ2
1
+
(x − µ2)2
2σ2
2
= ln
1
2πσ1σ2cte
(4.7.463)
and we get:
(x − µ1)2
2σ2
1 ln
1
2πσ1σ2cte
+
(x − µ2)2
2σ2
2 ln
1
2πσ1σ2cte
= 1 (4.7.464)
We recognize here the analytical equation of an ellipse (see section Analytical Geometric)!
A plot of iso-lines with µ =
3
2
, Σ =
25 0
0 9
give us:
Figure 4.82 – Plot of the iso-lines of the bivariate Normal function (non-correlated case)
But now recall that when we got:
f(X1, X2) =
1
2π|Σ|1/2
e
−
1
2
−−−−→
(x−µ)T Σ−1
−−−−→
(x−µ)
(4.7.465)

Draft
the variance-covariance matrix was zero everywhere except on the diagonal, implying verbatim
the independence of the two random variables. We can obviously guess that the generalization
is that the variance-covariance matrix is non-zero in the diagonal and then the two random
variables are correlated. Consequently, the iso-lines become with values such as µ =
3
2
, Σ =
10 5
5 5
:
Figure 4.83 – Plot of the iso-lines of the bivariate Normal function (correlated case)
So the correlation rotates the axis of the ellipses! Note that we have therefore:
Σ =
σ2
11 σ12
σ21 σ2
22
=
σ2
11 σ12
σ12 σ2
22
⇒ Σ−1
=
1
σ2
11σ2
22 − σ2
12
σ2
11 −σ12
σ21 σ2
22
(4.7.466)
and thus verbatim:
|Σ| = σ2
11σ2
22 − σ2
12 (4.7.467)
Recall that we saw during our study of the correlation coefﬁcient that (well... normally... the
R notation for the correlation is used only if the variances are estimated but as it is the most
common notation in practice we will still us it...):
σ12 = cov(X1, X2) = c(X1, X2) = RX1,X2 σ11σ22 (4.7.468)
Thus:
Σ−1
=
1
σ2
11σ2
22 − σ2
12
σ2
11 −σ12
σ21 σ2
22
=
1
σ2
11σ2
22(1 − R2
12)
σ2
11 −σ12
σ21 σ2
22
(4.7.469)
and the exponent of the exponential of the bivariate Normal takes a form that we can found very

Draft
often in the literature:
x1 − µ1
x2 − µ2
T
1
σ2
11σ2
22(1 − R2
12)
σ2
11 −σ12
σ21 σ2
22
x1 − µ1
x2 − µ2
=
x1 − µ1
x2 − µ2
T
1
σ2
11σ2
22(1 − R2
12)
σ2
11 −R12σ11σ22
−R12σ11σ22 σ2
22
x1 − µ1
x2 − µ2
=
σ2
22(x1 − µ1)2
+ σ2
11(x2 − µ2)2
− 2R12σ11σ22(x1 − µ1)(x2 − µ2)
σ2
11σ2
22(1 − R2
12)
=
1
1 − R2
12
x1 − µ1
σ11
2
+
x2 − µ2
σ22
2
− 2R12
x1 − µ1
σ11
x2 − µ2
σ22
(4.7.470)
Note that if the random variables are centered reduced, then we have:
Σ−1
=
1
1 − R2
12
(4.7.471)
and thus the exponent of the exponential of the bivariate Normal distribution becomes:
Σ−1
=
1
1 − R2
12
x2
1 + x2
2 − 2R12x1x2 (4.7.472)
Thus, the density function of the bivariate Normal centered reduced distribution will be written:
Now consider the important case in engineering, astronomy and quantum physics by returning
to the following notation:
f(x1, x2, R) =
1
2π(1 − R2
12)
e
−
x2
1 + x2
2 − 2R12x1x2
2(1 − R2
12
=
1
2π(1 − R2
12)
e−x2
1
1
2π(1 − R2
12)
e−x2
2 e
R11x1x2
1 − R2
12
= N(0, 1 − R2
12)N(0, 1 − R2
12)e
R11x1x2
1 − R2
12
= N(0, 1)N(0, 1) 1 − R2
12
2
e
R11x1x2
1 − R2
12
(4.7.473)
Thus, we can see that a bivariate Normal reduced centered distribution function normal can be
constructed by the multiplication of two Normal centered and reduced distributions themselves
multiplied by a term that depends mainly on the correlation parameter. The latter term includes
the nature of the dependence of the two random variables and provides the link between the
marginal distributions (both Normal centered and reduced) to obtain the joint bivariate Normal
distribution.
If necessary (this can be very useful in practice), here is the Maple 4.00b code to plot a bivariate
Normal function (taking the last example) even if it is also very simple to do with a spreadsheet
software like Microsoft Excel:
bfXa@xDyDrhoDmuIDmuPDsigm—IDsigm—PAEb@IG@PB€iBsqrt@sigm—IBsigm—PB@IErho”PAAAA
Bexp@@EIG@PB@IErho”PAAAB@@@xEmuIAGsqrt@sigm—IAA”PC@@yEmuPAGsqrt@sigm—PAA”P

Draft
EPBrhoB@@xEmuIAGsqrt@sigm—IAAB@@yEmuPAGsqrt@sigm—PAAAAY
bplotQd@f@xDyDSGsqrt@IHBSADQDPDIHDSADxaERFFIHDyaERFFWDgrida‘RHDRH“AY
and for the plot with the iso-lines:
bwith@plotsAX
b™ontourplot@f@xDyDSGsqrt@IHBSADQDPDIHDSADxaERFFIHDyaERFFWDgrida‘RHDRH“AY
and we can check that it is a probability density function by writing:
bint@int@f@xDyDSGsqrt@IHBSADQDPDIHDSADxaEinfinityFFFCinfinityA
DyaEinfinityFFFCinfinityAY
or calculate the cumulative probability between two intervals:
bev—lf@int@int@f@xDyDSGsqrt@IHBSADQDPDIHDSADxaEQFFFCRADyaESFFFCPAAY
4.7.6.9.4 Normal Reduced Centered Distribution
The Gauss-Laplace distribution is not tabulated as we must then have so many numerical tables
as possible values for the mean µ and standard deviation σ (which are the parameters of the
function as we have seen it).
Therefore, by a change of variable, the Normal distribution becomes the Normal reduced cen-
tered distribution more often named the standard Normal distribution where:
1. Centered refers to subtracting the mean µ to the measures (thus the distribution function
is symmetric to the vertical axis).
2. Reduced refers to the division by the standard deviation σ (thus the distribution function
has a unit variance).
By this change of variable, the variable k is replaced by the reduced centered random variable:
k∗
=
k − µ
σ
(4.7.474)
If the variable k has for mean µ and standard deviation σ then the variable k∗
has a mean of 0
and standard deviation of 1 (this last variable is usually denoted by the letter Z).
Thus the relation:
P(k, µ, σ) =
1
σ
√
2π
e
−
(k − µ)2
2σ2
(4.7.475)

Draft
is therefore written (trivially) more simply:
P(k∗
, 0, 1) =
1
√
2π
e
−
k∗2
2 (4.7.476)
which is just the explicit expression of the reduced centered Normal distribution (standard Nor-
mal) often denoted N(0, 1) which we will ﬁnd very often in the sections of physics, ﬁnance,
quantitative management and engineering!
Remark
Calculate the integral of the previous relation for an interval can not be done accurately
formally speaking. One possible and simple idea is then to express the exponential in
a Taylor series and then be integrated term by term of the series (making sure to take
enough terms for convergence!).
4.7.6.9.5 Henry’s Line
Often in business it is the Gauss-Laplace (Normal) distribution that is analyzed but com-
mon and easily accessible software like Microsoft Excel are unable to verify that the measured
data follow a Normal distribution when we do the frequency analysis (there are no default
integrated tool allowing users to check this assumption) and we do not have the original
ungrouped data.
The trick then is then to use the reduced centered variable that is build as we have see above
with the following relation:
k∗
=
k − µ
σ
(4.7.477)
The idea of the Henry’s Line is then to use the linear relation between k and k∗
given by the
equation of the line:
k∗
= f(k) =
k
σ
−
µ
σ
(4.7.478)

Draft
Example:
Suppose we have the following frequency analysis of 10,000 receipts in a supermarket:
Price Number Cumulated number Relative frequencies
of receipts of receipts of receipts of receipts
[0,50[ 668 668 0.068
[50,100[ 919 1,587 0.1587
[100,150[ 1,498 3,085 0.3085
[150,200[ 1,915 5,000 0.5000
[200,250[ 1,915 6,915 0.6915
[250,300[ 1,498 8,413 0.8413
[300,350[ 919 9,332 0.9332
[350,400[ 440 9,772 0.9772
[400 and + 228 10,000 1
Table 4.25 – Supermarket receipt amount distribution
If we now plot this in Microsoft Excel 11.8346 we get:
Figure 4.84 – Distribution of receipts amount
What looks terribly like a Normal distribution, thus the authorization, without too much
risk to use in this example the technique of Henry’s line.
But what can we do now? Well... now that we know the cumulative frequency, it
remains for us to calculate each k∗
using numerical tables or the xy‚wƒsx†@ A function
of Microsoft Excel 11.8346 (remember that formal integration of the Gaussian function
is not easy...).
This will give us the values of the standard Normal distribution N(0, 1) of these respec-
tive cumulative frequencies (cumulative distribution function). So we get (we leave to
the reader to take its statistic table or open its favorite software...):

Draft
Upper limit Cumulated Correspondance for k∗
of the interval relative frequencies of N(0, 1)
50 0.068 -1.5
100 0.1587 -1
150 0.3085 -0.5
200 0.5000 0
250 0.6915 0.5
300 0.8413 1
350 0.9332 1.5
400 0.9772 2
- 1 -
Table 4.26 – Cumulative relative frequencies to the Henry’s line
Note that in the type of table above, in Microsoft Excel, the null and unit cumulative
frequencies will generated some errors. You should then play a little bit...
As we speciﬁed earlier, we have under discrete form:
k∗
= f(ki) =
ki
σ
−
µ
σ
(4.7.479)
So graphically in Microsoft Excel 11.8346 we can thanks to our table plot the following
chart (obviously we could do strictly a linear regression in the rules of art as seen in the
chapter of Numerical Methods with conﬁdence, prediction intervals and other stuffs...):
Figure 4.85 – Linearized form of the distribution
So thanks to the linear regression given by Microsoft Excel 11.8346 (or calculated by you
using the techniques of linear regressions seen in the chapter on Numerical Methods). It
comes:
k∗
= f(k) =
k
σ
−
µ
σ
= 0.01k − 2 (4.7.480)

Draft
we immediately deduce that:
σ = 100 µ = 200 (4.7.481)
This is thus a particular technique for a particular distribution! Similar techniques more
or less simple (or complicated depending on the case...) exist for others distributions.
See now another approximate approach to solve this problem. Let take us again our table
for this example:
Price Upper limit Center Relative cumulative
of receipts of the interval frequencies in %
[0,50[ 50 25 6.8
[50,100[ 100 75 15.87
[100,150[ 150 125 30.85
[150,200[ 200 175 50.00
[200,250[ 250 225 69.15
[250,300[ 300 275 84.13
[300,350[ 350 325 93.32
[350,400[ 400 375 97.72
[400 and + - - 100
The average is now calculated using the central value of the intervals and sample sizes
according to the relation we have seen at the beginning of this section:
ˆµ =
N
i=1 nixi
N
i=1 ni
(4.7.482)
Price Center Relative cumulative Calculation
of receipts frequencies in %
[0,50[ 25 668 16,700
[50,100[ 75 919 68,925
[100,150[ 125 1,498 187,250
[150,200[ 175 1,915 335,125
[200,250[ 225 1,915 430,875
[250,300[ 275 1,498 411,950
[300,350[ 325 919 411,950
[350,400[ 375 440 165,000
[400 and + - - -
Sum: 9,772 1,914,500
Average:
1, 914, 500
9, 772
= 195.92
The average that we have calculated yet is also quite close to the average obtained previ-
ously with the Henry’s line.

Draft
The standard deviation will now be calculated using also the central value of the intervals
and sample sizes according to the relation seen at the beginning of this chapter:
ˆσ =
N
i=1
ni(xi − ˆµ)
N
i=1
ni
(4.7.483)
Price Center Relative cumulative Calculation
of receipts frequencies in %
[0,50[ 25 668 16,700
[50,100[ 75 919 68,925
[100,150[ 125 1,498 187,250
[150,200[ 175 1,915 335,125
[200,250[ 225 1,915 430,875
[250,300[ 275 1,498 411,950
[300,350[ 325 919 411,950
[350,400[ 375 440 165,000
[400 and + - 228 -
Variance: 8364.16
Standard Deviation: 91.45
The standard deviation that we have calculated yet is also quite close to the standard
deviation obtained with the method of the Henry’s line.
4.7.6.9.6 Q-Q plot
Another way to judge of the quality of ﬁt of experimental data with a theoretical distri-
bution (whatever that is!) is the use of a quantile-quantile plot or simply called q-q
plot.
The idea is pretty simple, it based on the comparison the experimental data relatively to the
theoretical data that are supposed to follow a particular distribution. Thus, in the case of our
example, if we take the values of the mean (∼ 200) and standard deviation (∼ 100) obtained
with the Henry’s line as theoretical parameters for the Normal distribution, we get:

Draft
Price Upper experimental Relative cumulative Upper theoretical
of receipts limit (imposed) frequencies in % 50.91
[0,50[ 50 6.80% 50.91
[50,100[ 100 15.87% 100.02
[100,150[ 150 30.85% 149.99
[150,200[ 200 50.00% 200
[200,250[ 250 69.15% 250.01
[250,300[ 300 84.13% 299.98
[300,350[ 350 93.32% 350.00
[350,400[ 400 97.72% 399.90
[400 and + - 100% -
Plotted, this gives us the famous Q-Q plot:
Figure 4.86 – Q-Q plot of the distribution
And of course we can compare the observed quantiles with the supposed theoretical distribution.
More the points will be aligned on the line of unit slope and zero intercept origin, the better will
be the ﬁt! It’s very visual, very simple and widely used by non-specialists in business statistics.

Draft
4.7.6.10 Log-Normal Distribution
We say that a positive random variable X follows a log-normal function (or log-normal
distribution) if by writing:
y = ln(x) (4.7.484)
we see that y follows a Normal distribution of mean µ and variance σ2
(moments of the Normal
distribution).
Verbatim by the properties of logarithms, a variable can be modeled by a log-normal distribution
if it results of the multiplication of many small independent factors (property of the product in
sum of the logarithms and stability of the Normal distribution by the addition).
The density function of X for x ≥ 0 is then (see section Differential And Integral Calculus):
f(x) =
1
σx
√
2π
e
−
(ln(x) − µ)2
2σ2 (4.7.485)
that can be calculated in Microsoft Excel 11.8346 with the vyqxy‚whsƒ„@ A function or its
inverse by vyqsx†@ A.
This type of scenario is frequent in physics, in technical maintenance or ﬁnancial markets in the
options pricing model (see the respective sections of the book for various application examples).
There is also an important remark with respect to the log-normal distribution further when we
will develop the central limit theorem!
Let us show that the cumulative probability function corresponds to a Normal distribution if we
make the change of variables mentioned above:
+∞ˆ
0
f(x)dx =
+∞ˆ
0
1
σx
√
2π
e
−
(ln(x) − µ)2
2σ2
dx =
1
σ
√
2π
+∞ˆ
0
1
x
e
−
(ln(x) − µ)2
2σ2
dx (4.7.486)
by writing:
y = ln(x) ⇒
dy
dx
=
1
x
⇔ dx = xdy (4.7.487)
and (by deﬁnition):
x = ey
(4.7.488)
we then get:
+∞ˆ
0
f(x)dx =
1
σ
√
2π
+∞ˆ
0
1
x
e
−
(ln(x) − µ)2
2σ2
dx =
1
σ
√
2π
+∞ˆ
−∞
1
x
e
−
(y − µ)2
2σ2
xdy
=
1
σ
√
2π
+∞ˆ
−∞
e
−
(y − µ)2
2σ2
dy
(4.7.489)

Draft
So we found again the Normal distribution!
The mean (average) of X is then given by (the natural logarithm being not deﬁned for x 0
we start the integral from zero):
E(X) =
+∞ˆ
0
xf(x)dx =
1
σ
√
2π
+∞ˆ
0
x
1
x
e
−
(ln(x) − µ)2
2σ2
dx =
1
σ
√
2π
+∞ˆ
0
e
−
(ln(x) − µ)2
2σ2
dx
=
1
σ
√
2π
+∞ˆ
−∞
e
−
(u − µ)2
2σ2
+u
du
(4.7.490)
where we performed the change of variable:
u = ln(x) ⇔ x = eu
xdu = dx (4.7.491)
The expression:
(u − µ)2
2σ2
+ u (4.7.492)
moreover being equal to:
−
1
2σ2
u − (µ + σ2
)
2
− µ + σ2 2
+ µ2
(4.7.493)
the last integral also becomes:
E(X) =
e
(µ + σ2
)
2
− µ2
2σπ
σ
√
2π
+∞ˆ
−∞
e
−
1
2σπ
u−(µ+σ2
)
2
du
= e
(µ + σ2
)
2
− µ2
2σπ = e
µ+
σ2
2
(4.7.494)
and where we used the property that emerged during our study of the Normal distribution, that
is to say that any integral of the form:
+∞ˆ
−∞
e
−
(x − cte
)2
2σ2
dx = σ
√
2π (4.7.495)
always has the same value!
To calculate the variance, recall that for a random variable X, we have the Huygens theorem:
V(X) = E(X2
) − E(X)2
(4.7.496)

Draft
Let us calculate E(X2
) by performing similarly to previous developments:
E(X2
) =
+∞ˆ
0
x2
f(x)dx =
1
σ
√
2π
+∞ˆ
0
xe
−
(ln(x) − µ)2
2σ2
dx
=
1
σ
√
2π
+∞ˆ
−∞
eu
e
−
(u − µ)2
2σ2
eu
du =
1
σ
√
2π
+∞ˆ
−∞
e
−
(u − µ)2
2σ2
+2u
du
=
1
σ
√
2π
+∞ˆ
−∞
e
−
(u − (µ + 2σ2
))
2
2σ2
e
(µ + 2σ2
)
2
− µ2
2σ2
du
=
e
(µ + 2σ2
) − µ2
2σ2
σ
√
2π
+∞ˆ
−∞
e
−
(u − (µ + 2σ2
))
2
2σ2
du
= e
(µ + 2σ2
) − µ2
2σ2
= e
4µσ2
+ 4σ4
2σ2
= e2µ+2σ2
= e2(µ+σ2)
(4.7.497)
where once again we have the change of variable:
u = ln(x) ⇔ x = eu
⇒ dx = eu
du (4.7.498)
and where we transformed the expression:
−
(u − µ)2
2σ2
+ 2u (4.7.499)
as:
−
1
2σ2
u − (µ + 2σ2
)
2
− µ + 2σ2 2
+ µ2
(4.7.500)
Then:
V(X) = E(X2
) − E(X)2
= e2µ+2σ2
−


e
µ+
σ2
2



2
= e2µ+2σ2
− e2µ+σ2
= e2µ+σ2
eσ2
− 1
(4.7.501)
Here is a plot example of the distribution and cumulative distribution of the Log-Normal func-
tion of parameters (µ, σ) = (0, 1):

Draft
Figure 4.87 – Log-Normal law (mass and cumulative distribution function)
4.7.6.11 Continuous Uniform Distribution
Let us choose a b. We define the continuous uniform distribution function or uniform
function by the relation:
Ua,b(x) =
1
b − a
1[a,b] (4.7.502)
where 1[a,b] means that outside the domain of definition [a, b] the distribution function is zero.
We will find this type of notation later in some other distribution functions.
So we have for the cumulative distribution function:
P(X ≤ x) =
xˆ
a
1
b − a
1[a,b]dx =
1
b − a
1[a,b]
xˆ
a
dx =
x[a,b] − 1
b − a
(4.7.503)
It is indeed a distribution function because it satisfies (simple integral):
+∞ˆ
−∞
Pa,bdx =
+∞ˆ
−∞
1
b − a
1[a,b]dx =
1
b − a
1[a,b]
bˆ
a
dx =
1
b − a
1[a,b]x|b
a = 1[a,b]
b − a
b − a
= 1
(4.7.504)
The continuous uniform function has for expected mean:
µ = E(X) =
bˆ
a
xf(x)dx =
1
b − a
bˆ
a
dx =
1
b − a
x2
2
b
a
=
1
b − a
b2
− a2
2
=
1
b − a
(b + a)(b − a)
2
=
a + b
2
(4.7.505)

Draft
and for the variance using the Huygens theorem:
V(X) = E(X2
) − E(X)2
=
bˆ
a
x2
f(x)dx −
a + b
2
2
=
1
b − a
bˆ
a
x2
dx −
a + b
2
2
=
1
b − a
x3
3
b
a
−
a + b
2
2
=
1
b − a
b3
− a3
3
−
a + b
2
2
=
1
b − a
(b − a)(b2
+ ab + a2
3
−
a + b
2
2
=
1
3
(b2
+ ab + a2
) −
1
4
(b2
+ 2ab + a2
) =
4(b2
+ ab + a2
) − 3(b2
+ 2ab + a2
)
12
=
4b2
+ 4ab + 4a2
− 3b2
− 6ab − 3a2
)
12
=
b2
− 2ab − a2
12
=
(b − a)2
12
(4.7.506)
Here is a plot example of the distribution and cumulative distribution of the continuous uniform
function of parameters (a, b) = (0, 1):
Figure 4.88 – Uniform continuous law (mass and cumulative distribution function)
Remark
This function is often used in business simulation to indicate that the random variable
has equal probabilities to have a value within a certain interval (typically in portfolio
returns or in the estimation of project durations). The best example of application is
again CrystalBall or @Risk software that integrate with Microsoft Project.
Let us see an interesting result of the continuous uniform distribution (and that applies also to
the discrete one as well...).
I often hear managers (who consider themselves at high level) that if we have a measure with
an equal probability to occur in a closed given interval, then the sum of two such independent
random variables have also the same equal probability in the same interval!

Draft
Now we will prove here that this is not the case (if someone has a more elegant proof I’m
interested)!
Proof 4.27.3. Consider two independent random variables X and Y that follow a uniform dis-
tribution in a closed interval [0, a]. We are searching the density of their sum will be written:
Z = X + Y (4.7.507)
Then we have:
fX(x) = fY (y) =
+1 if 0 ≤ x, ya
0 otherwise
(4.7.508)
(4.7.509)
with the variable:
0 ≤ z ≤ 2a (4.7.510)
To calculate the distribution of the sum, remember that we know that in discrete terms this is
equivalent to the joint product of probabilities (see section Probabilities) of the occurrence of
two continuous variables (remember the same kind of calculation in the discreet form!).
That is to say:
fZ(z) =
+∞ˆ
−∞
fX(z − y)fY (y)dy (4.7.511)
As fY (y) = 1 if 0 ≤ y ≤ a and 0 otherwise then the product of the previous convolution
reduces to:
fZ(z) =
aˆ
0
fX(z − y)dy (4.7.512)
The integrand is by deﬁnition 0 except by construction in the interval 0 ≤ z − y ≤ a it is then
1.
Let us focus on the limits of the integral that is in this case the only one that is interesting ....
First we make a change of variables by writing:
u = z − y (4.7.513)
thus:
du = −dy (4.7.514)
The integral can be then written in this interval after the change of variable:
fZ(z) =
aˆ
0
fX(z − y)dy = −
z−aˆ
z
fX(u)du =
z−aˆ
z
du (4.7.515)

Draft
Remembering that we have seen at the beginning that 0 ≤ z ≤ 2a, then we have immediately
if z 0 and z 2a that the integral is zero.
We will consider two cases for the interval because the convolution of these two rectangular
functions can be distinguished according to the situation where at first they cross (nest), that is
to say where 0 ≤ z ≤ a, and then recede from each other, that is to say a z ≤ 2a.
• In the first case (nest) where 0 ≤ z ≤ a:
fZ(u) =
zˆ
z−a
du =
zˆ
0
du = u|z
0 = z (4.7.516)
where we changed the lower bound to 0 because anyway fX(u) is zero for any negative
value (and when 0 ≤ z ≤ a,z − a is precisely zero or negative!).
• In the second case (dislocation) where a z ≤ 2a:
fZ(u) =
zˆ
z−a
du =
aˆ
z−a
du = a − (z − a) = 2a − z (4.7.517)
where we changed the upper terminal a because anyway fX(u) is zero for any higher
value (and when a ≤ z ≤ 2a, z is just larger than a).
So in the end, we have:
fZ(z) =



z if 0 ≤ z ≤ a
2a − z if a ≤ z ≤ 2a
0 otherwise
(4.7.518)
(4.7.519)
Q.E.D.
This is a particular case, deliberately simplified, of the triangular distribution that we will dis-
cover just after...
This result (which may seems perhaps not intuitive) can be check in a few seconds with a spread-
sheet software like Microsoft Excel 11.8346 using the ‚exhfi„‡iix@A and the p‚i…ixg‰@ A
functions.

Draft
4.7.6.12 Triangular Distribution
Let a c b.We define the triangular distribution (or triangular function) by construction
based on the following two distribution functions:
Pa,c(x) =
2(x − a)
(b − a)(c − a)
· 1[a,c] and Pc,b(x) =
2(b − x)
(b − a)(b − c)
· 1]c,b] (4.7.520)
where a is often assimilated with the optimistic value, c to the modal value and b the pessimistic
value.
It is also the only way to write this distribution function if the reader keeps in mind that the base
of a triangle of lenght c − a must have a height h equal to 2/(c − a) as its total area is equal to
unity (we will soon prove it).
Here is a plot example of the triangular distribution and cumulative distribution for the param-
eters (a, c, b) = (0, 3, 5):
Figure 4.89 – Triangular law (mass and cumulative distribution function)
The slope of the first straight line (increasing from left) is obviously:
2a
(b − a)(c − a)
(4.7.521)
and the slope of the second straight line (decreasing to the right):
−2
(b − a)(b − c)
(4.7.522)
This function is a distribution function if it satisfies:
ρ =
+∞ˆ
−∞
(Pa,c(x) + Pc,b(x))dx = 1 (4.7.523)

Draft
It is in this case, simply the area of the triangle which we recall is simply the base multiplied by
the height divided by 2 (see section Geometric Shapes):
ρ =
1
2
(b − a) ·
2
b − a
= 1 (4.7.524)
Remark
This function is widely used in project management in the context of task duration esti-
mations or in industrial simulations. Where a corresponds to the optimistic value, c to
the expected value (mode) and the value b to the pessimistic value. The best example
of application is again the softwares CrystalBall or @Risk that are add-ins for Microsoft
Project.
The triangular function has also for mean (average):
µ =
+∞ˆ
−∞
xf(x)dx =
cˆ
a
x
2(x − a)
(b − a)(c − a)
dx +
bˆ
c
x
2(b − x)
(b − a)(b − c)
dx
=
2
(b − a)(c − a)
1
3
x3
−
1
2
ax2
|c
a +
2
(b − a)(c − a)
1
2
bx2
−
1
3
x3
|b
c
=
2
(b − a)(c − a)
1
3
c3
−
1
2
ac2
−
1
3
a3
−
1
2
a3
−
2
(b − a)(c − a)
1
2
b3
−
1
3
b3
−
1
2
bc2
−
1
3
c3
= 2
(b − c)
1
3
c3
−
1
2
ac2
+
a3
6
+ (c − a)
1
6
b3
−
1
2
bc2
−
1
3
c3
(b − a)(c − a)(b − c)
= 2
1
3
bc3
−

1
2
abc2
+
ba3
6
−

1
3
c4
+
1
2
ac3
−

a3
c
6
(b − a)(c − a)(b − c)
+ 2
1
6
cb3
−
1
2
bc3
+

1
3
c4
−
ab3
6
+

1
2
abc2
−
1
3
ac3
(b − a)(c − a)(b − c)
=
2
3
bc3
+ ac3
−
1
3
a3
c +
1
3
cb3
+
ba3
3
−
ab3
3
− bc3
−
2
3
ac3
(b − a)(c − a)(b − c)
=
−
1
3
bc3
−
1
3
a3
c +
1
3
ba3
−
1
3
1
3
ab3
+
1
3
cb3
+
1
3
ac3
(b − a)(c − a)(b − c)
=
1
3
−bc3
− a3
c + ba3
− ab3
+ cb3
+ ac3
(b − a)(c − a)(b − c)
=
1
3
(a + b + c)(b − a)(c − a)(b − c)
(b − a)(c − a)(b − c)
=
a + b + c
3
(4.7.525)

Draft
and for variance:
σ2
=
+∞ˆ
−∞
(x − µ)2
f(x)dx =
cˆ
a
(x − µ)2 2(x − a)
(b − a)(c − a)
dx +
bˆ
c
(x − µ)2 2(b − x)
(b − a)(b − c)
dx
= −
1
6
c (−3c3
+ 8c3
µ + 4c2
a − 6cµ2
− 12cµa + 12µ2
a)
(b − a)(c − a)
+
1
6
a2
(a2
− 4µa + 6µ2
(b − a)(c − a)
= −
1
6
−3c3
+ 8c2
µ + 4c2
b − 6cµ2
− 12cµb + 12µ2
b
(b − a)(c − a)
+
1
6
b2
(b2
− 4µb + 6µ2
)
(b − a)(c − a)
=
1
6
b2
+
1
6
ba −
2
3
µb +
1
6
a2
−
2
3
µa + µ2
+
1
6
ca −
2
3
cµ +
1
6
c2
(4.7.526)
We can replace µ by the result obtained before and we get after simplification (it is boring
algebra...):
σ2
=
a2
+ b2
+ c2
− ab − ac − bc
18
(4.7.527)
We can show that the sum of two independent random variables, each uniformly distributed on
[a, b] (i.e. independent and identically distributed) follows a triangular distribution on [2a, 2b]
but if they do not have the same limits, then their sum gives something that has no name to my
knowledge...
4.7.6.13 Pareto Distribution
The Pareto distribution (or Pareto law), also named power law or scale law is the for-
malization of the 80−20 principle. This decision tool helps determine the critical factors (about
20%) influencing the majority (80%) of the goal.
Remark
This distribution is a fundamental and basic tool in quality management (see Industrial
Engineering and Quantitative Management sections). It is also used in reinsurance. The
theory of queues had also some interest in this distribution when some research in the
1990s showed that this distribution also seems ton explain well a number of variables
observed in the Internet traffic (and more generally on all high speed data networks).
A random variable is said by definition follow a Pareto distribution if its cumulative distribution
function is given by:
P(X x) = 1 −
xm
x
k
(4.7.528)
with x that must be greater than or equal to xm.
The Pareto density function (distribution function) is then given by:
f(x) =
d
dx
1 −
xm
x
k
= −
d
dx
xm
x
k
= −xk
m
d
dx
1
xk
= k
xk
m
xk+1
(4.7.529)

Draft
with k ∈ R+ and x ≥ xm ≥ 0 (then x 0). The Pareto distribution is deﬁned by two param-
eters, xm and k (Named Pareto index). This distribution is also said to be scale invariant or
fractal distribution, because of the following property:
f cte
· x = kxk
m cte
· x
−k−1
= cte −k−1
kxk
mx−k−1
= cte −k−1
f(x) ∝ f(x) (4.7.530)
The Pareto function is also well a distribution function as the cumulative distribution function
known we have:
+∞ˆ
0
f(x)dx = 1 −
xm
x
k +∞
xm
= 1 −
xm
+∞
k
− 1 −
xm
xm
k
= (1 − 0) − (1 − 1k
) = 1
(4.7.531)
The expected mean is given by:
µ = E(X) =
+∞ˆ
xm
xf(x)dx =
+∞ˆ
xm
xk
xk
m
xk+1
x = kxk
m
+∞ˆ
xm
x
1
xk
x = −
kxk
m
k − 1
1
xk−1
+∞
xm
=
kxm
k − 1
(4.7.532)
if k 1. If k ≤ 1, the mean does not exist. To calculate the variance, using the Huygens
theorem:
V(X) = E(X2
) − E(X)2
(4.7.533)
we get:
E(X2
) =
+∞ˆ
xm
x2
f(x)x = kxk
m
+∞ˆ
xm
1
xk−1
x = −
kxk
m
k − 2
1
xk−2
+∞
xm
=
kx2
m
k − 2
(4.7.534)
if k 2. If k ≤ 2, E(X2
) doesn’t exists. So if k 2:
σ2
= V(X) =
kx2
m
k − 2
−
kxm
k − 1
2
=
kx2
m
(k − 1)2(k − 2)
(4.7.535)
If k ≤ 2, the variance doesn’t exists. Here is a plot example of the Pareto distribution and
cumulative distribution for the parameters (x, xm, k) = (x, 1, 2):

Draft
Figure 4.90 – Pareto law (mass and cumulative distribution function)
Remark
See that when k → +∞ the distribution approaches δ(x−xm) where δ is the Dirac delta
function.
There is another important way to deduce the family of Pareto distributions that allows us to
understand many things about other distributions and that is often presented as follows:
Let us write x0 the threshold beyond which we calculate the mean of the considered quantity,
and E(Y ) the mean beyond this threshold x0 as it is proportional (linearly dependent) to the
chosen threshold:
E(Y ) = ax0 + b (4.7.536)
This functional relation expresses the idea that the conditional mean beyond the threshold x0 is
a multiple of this threshold plus a constant, that is to say a linear function of the threshold.
Thus, in project management, for example, we could say that once a certain threshold of time
is exceeded, the expected duration is a multiple of this threshold plus a constant.
If a linear relationship of this type exists and is satisﬁed, then we talk about a probability distri-
bution in the form of a generalized Pareto distribution.
Consider the mean of the Bayesian conditional function given by (see section Probabilities):
E(Y ) =
1
P(X ≥ x0)
+∞ˆ
x0
yf(y)dy (4.7.537)
If we write F(y) the cumulative distribution function f(y), then we have by deﬁnition:
dF(y) = f(y)dy (4.7.538)

Draft
Thus:
E(Y ) =
1
P(X ≥ x0)
+∞ˆ
x0
ydF(y) (4.7.539)
and if we define:
¯FX(x) = P(X ≥ x0) = 1 − FX(x) (4.7.540)
what we can assimilate to the tail of the distribution.
We get:
E(Y ) =
1
¯F(x0)
+∞ˆ
x0
ydF(y) (4.7.541)
and therefore we seek the very special case where:
E(Y ) =
1
¯F(x0)
+∞ˆ
x0
ydF(y) = ax + b (4.7.542)
this is to say:
+∞ˆ
x0
ydF(y) = (ax + b) ¯F(x0) (4.7.543)
Differentiating with respect to x, we find:
d
dx


+∞ˆ
x0
ydF(y)

 =
d
dx
(ax + b) ¯F(x0) (4.7.544)
The derivative of the integral defined above will be the derivative of a constant (valorisation of
the integral in +∞) minus the derivative of the analytical expression of the integral for x0. So
we have:
d
dx


+∞ˆ
x0
ydF(y)

 =
d
dx


+∞ˆ
x0
yf(y)dy

 = −xf(x) = −x
dF(x)
dx
= a ¯F(x) + (ax + b)
d ¯F(x)
dx
(4.7.545)
Thus:
−x
dF(x)
dx
= a ¯F(x) + (ax + b)
d ¯F(x)
dx
(4.7.546)
and as:
dF = d(1 − ¯F) = −d ¯F (4.7.547)

Draft
it comes:
x
d ¯F
dx
= a ¯F(x) + (ax + b)
d ¯F
dx
(4.7.548)
After simpliﬁcation and rearrangement we obtain:
a ¯F(x)dx = − (x(a − 1) + b) d ¯F(x) (4.7.549)
which is a differential equation in ¯F(x). Its Resolution provides all forms of seek Pareto distri-
butions, according to the values taken by the parameters a and b.
To solve this differential equation, consider the special case where a 1, b = 0. Then we have:
a ¯F(x)dx = −x(a − 1)d ¯F(x) (4.7.550)
By writing:
k =
a
a − 1
(4.7.551)
We then get:
−
1
x
dx =
1
k
1
¯F(x)
d ¯F(x) (4.7.552)
and therefore:
− ln(x) =
1
k
ln( ¯F(x)) + cte
(4.7.553)
It comes:
e
ln


1
x


= e
1
k
ln(¯x)+cte
= e
1
k
ln( ¯F(x))
ecte
(4.7.554)
and therefore:
ln
1
x
= ¯F(x)
1
k cte
(4.7.555)
We have:
¯F(x) =
cte
x
k
=
xm
x
k
(4.7.556)
Then it comes form the cumulative distribution function:
F(x) = 1 − ¯F(x) = 1 −
xm
x
k
(4.7.557)
If we seek for the distribution function, we derive by x to get:
f(x) =
dF(x)
dx
= k
xk
m
xk+1
(4.7.558)

Draft
This is the Pareto distribution we have used since the beginning and called Pareto distribution
of type I (we won’t see in this book those of type II).
An interesting thing to observes is the case of the resolution of the following differential equa-
tion:
a ¯F(x)dx = −(x(a − 1) + b)d ¯F(x) (4.7.559)
when a = 1, b 0. The differential equation is then reduced to:
¯F(x)dx = −bd ¯F(x) (4.7.560)
Thus:
−
1
b
dx =
1
¯F(x)
d ¯F(x) (4.7.561)
After integration:
−
1
b
x = ln( ¯F(x)) (4.7.562)
and therefore:
¯F(x) = e
−
1
b
x
(4.7.563)
If we make a small change in notation:
¯F(x) = e−λx
(4.7.564)
and that we write the distribution function:
F(x) = 1 − ¯F(x) = 1 − e−λx
(4.7.565)
and by derivating we get the distribution function of the exponential distribution:
F(x) = λe−λx
(4.7.566)
So the exponential distribution has a conditional mean threshold that is equal to:
E(Y ) = ax0 + b =
a=1
x0 + b =
b=
1
λ
x0 +
1
b
= x0 + σ (4.7.567)
So the conditional mean threshold is equal to itself plus the standard deviation of the distribu-
tion.

Draft
4.7.6.14 Exponential Distribution
We deﬁne the exponential distribution (or exponential law) by the following distribution
function:
P(x) = λe−λx
1[0,+∞] (4.7.568)
with λ 0 that as we will immediately see is in the fact that the inverse of the mean and where
x is a random variable without memory. This law is also sometimes denoted E(λ).
In fact the exponential distribution naturally appears from simple developments (see the Nu-
clear Physics chapter for example) under assumptions that impose a constance in the aging of
phenomenon. In the section of Quantitative Management, we have also proved in detail in the
section on the theory of queues, that this law was without memory. That is to say, that the
cumulative probability of a phenomenon occurs between the time t and t+s, if it is not realized
before, is the same as that the cumulative probability of occurring between the time 0 and s.
Remarks
R1. This function is occuring frequently in nuclear physics (see chapter of the same
name) or quantum physics (see also chapter of the same name) as well in reliability (see
Industrial Engineering) or in the theory of queues (see section Quantitative Management).
R2. We can get this distribution in Microsoft Excel 11.8346 with the iˆ€yxhsƒ„@ A
function.
It is also really a distribution function because it veriﬁes:
+∞ˆ
−∞
Pλ(x)dx = λ
+∞ˆ
0
e−λx
dx = λ −
1
λ
e−λx +∞
0
= − e−∞x
− e0x
= −(0 − 1) = 1
(4.7.569)
The exponential distribution has for expected mean using integration by parts:
µ =
+∞ˆ
−∞
xPλ(x)dx = λ
+∞ˆ
0
xe−λx
dx = − xe−λx +∞
0
−
+∞ˆ
0
−e−λx
dx =
+∞ˆ
0
e−λx
dx
= −
1
λ
e−λx +∞
0
=
1
λ
(4.7.570)
and for variance using once again the Huygens relation:
V(X) = E(X2
) − E(X)2
(4.7.571)
it remains for us to only the to calculate:
E(X2
) =
+∞ˆ
0
λx2
e−λx
x (4.7.572)

Draft
A variable change y = λ leads us to:
E(X2
) =
1
λ2
+∞ˆ
0
y2
e−y
dy (4.7.573)
A double integration by parts gives us:
bˆ
a
f(t)g (t)dt = f(t)g(t)|b
a −
bˆ
a
f (t)g(t)dt
+∞ˆ
0
y2
e−y
dy = −y2
e−y
|−∞
0 + 2
+∞ˆ
0
ye−y
dy = 2 −ye−y
|+∞
0 + 2
+∞ˆ
0
e−y
dy = 2
=
(4.7.574)
Hence:
E(X2
) =
2
λ2
(4.7.575)
we have therefore:
V(X) = E(X2
) − E(X)2
=
2
λ2
−
1
λ
2
=
1
λ2
(4.7.576)
So the standard deviation (square root of the variance for recall) and mean have exactly the
same expression!
Here is a plot example of the exponential distribution and cumulative distribution for the pa-
rameter λ = 1:
Figure 4.91 – Exponential law (mass and cumulative distribution function)
Now let us determine the distribution function of the exponential law:
P(X x) =
xˆ
0
λe−λt
dt = λ
xˆ
0
e−λt
dt = λ −
1
λ
e−λt
x
0
= 1 − e−λx
(4.7.577)

Draft
Remark
We will see later that the exponential distribution is a special case of a more general
distribution which is the chi-square distribution, the chi-square is also a special case of
a more general distribution that is the Gamma distribution. This is a very important
property used in the Poisson test for rare events (see also below).
4.7.6.15 Cauchy Distribution
Let X, Y be two independent random variables following a Normal reduced centered distribu-
tion (with zero mean and unit variance). Thus the density function is given for each variable
by:
fX(x) = fY (x) =
1
√
2π
e−x2
2 (4.7.578)
The random variable:
T =
X
|Y |
(4.7.579)
(the absolute value will be useful in an integral during a change of variable) follows a charac-
teristic appearance named Cauchy distribution (or Cauchy law) or even Lorentz law.
Let us now determine its density function f. To do this, recall that f is determined by the
(general) relation:
∀t ∈ R, P(T t) =
tˆ
−∞
f(x)dx (4.7.580)
So (application of elementary differential calculus):
f(t) =
d
dt
P(T ≤ t) (4.7.581)
in the case where f is continuous.
Since X and Y are independent, the density function of the random vector is given by one of
the axioms of probabilities (see section Probabilities):
(x, y) → fX(x) · fY (x) (4.7.582)
therefore:
P(T t) = P
X
|Y |
t = P(X t|Y |) =
ˆ
D
fX(x) · fY (x)dxdy (4.7.583)
where D = {(x, y)|x t|y|}. This last integral becomes:
ˆ
D
fX(x) · fY (x)dxdy =
+∞ˆ
−∞
t·|y|ˆ
−∞
fX(x) · fY (x)dxdy (4.7.584)

Draft
Let us make the following change of variables x = u|y| in the inner integral. We obtain:
P(T t) =
+∞ˆ
−∞
tˆ
−∞
fX(u · |y|) · fY (y)|y|dudy =
tˆ
−∞
+∞ˆ
−∞
fX(u · |y|) · fY (y)|y|dydu
(4.7.585)
Therefore:
f(t) =
d
dt
P(T t) =
+∞ˆ
−∞
fX(t · |y|) · fY (y)|y|dy =
1
2π
+∞ˆ
−∞
e−
y2(t2+1)
2 |y|dy (4.7.586)
Now the absolute value will be useful to write:
f(t) =
1
2π
+∞ˆ
−∞
e−
y2(t2+1)
2 |y|dy = −
1
2π
0ˆ
−∞
e−
y2(t2+1)
2 ydy +
1
2π
+∞ˆ
0
e−
y2(t2+1)
2 ydy (4.7.587)
For the ﬁrst integral we have:
1
2π
0ˆ
− inf
e−
y2(t2+1)
2 ydy =
e−
y2(t2+1)
2
t2 + 1
0
−∞
= −
1
t2 + 1
+
1
t2 + 1
= 0 (4.7.588)
It remains therefore only the second integral and making the change of variable v = y2
, we get:
f(t) =
1
2π
+∞ˆ
0
e−
ν(t2+1)
2 dν = −
e−
ν(t2+1)
2
π(t2 + 1)
+∞
0
=
1
π(t2 + 1)
(4.7.589)
What we will denote thereafter (to respect the notations adopted so far):
P(x) =
1
π(x2
+ 1) (4.7.590)
and that is just simply the so called Cauchy distribution. It is also a effectively a distribution
function because it veriﬁes (see section Differential and Integral Calculus):
+∞ˆ
−∞
P(x)dx =
1
π
+∞ˆ
−∞
1
x2 + 1
dx =
1
π
(arctan(+∞) − arctan(−∞)) =
1
π
π
2
− −
π
2
= 1
(4.7.591)
It is obvious that we get therefore for the cumulative distribution function:
P(X ≤ x) =
xˆ
−∞
P(x)dx =
1
π
xˆ
−∞
1
x2 + 1
dx =
1
π
(arctan(x) − arctan(−∞)) =
=
1
π
arctan(x) − −
π
2
=
1
π
arctan(x) +
1
2
(4.7.592)
Here is plot example of the Cauchy distribution:

Draft
Figure 4.92 – Cauchy law (mass function)
The Cauchy distribution has for expected mean:
µ =
+∞ˆ
−∞
xP(x)dx =
1
π
+∞ˆ
−∞
x
1 + x2
dx =
1
π


0ˆ
−∞
x
1 + x2
dx +
+∞ˆ
0
x
1 + x2
dx


=
1
2π
ln(1 + x2
)
0
−∞
+ ln(1 + x2
)
+∞
0
=
1
2π
(− ln(∞) + ln(∞)) = 0
(4.7.593)
Caution!!!! The above calculations do not give zero in facts because the subtraction of infinite
is not zero but indeterminate! The Cauchy distribution therefore and strictly speaking does not
admits an expected mean!
Thus, even if we can build a variance:
σ2
=
+∞ˆ
−∞
(x − µ)2
f(x)dx =
+∞ˆ
−∞
x2
P(x)dx =
1
π
+∞ˆ
−∞
x2
1 + x2
dx =
1
π
+∞ˆ
−∞
1 −
1
1 + x2
dx
=
2
π
lim
t→+∞
tˆ
0
1 −
1
1 + x2
dx = 2π lim
t→+∞
(t − arctan(t)) = +∞
(4.7.594)
this is absurd and does not exist strictly speaking as the mean doesn’t exists...!
The Cauchy distribution is used a lot in financial engineering as it is heavy tailed and therefore
a very good candidate to be more accurate in predicting extreme values at the opposite to the
Normal distribution that has the tails decreasing to quick. Further the Cauchy distribution is a
heavy tailed law with a support on R when the Pareto distribution (also heavy tailed) is defined
only on R+
.

Draft
The Cauchy distribution if one of the most famous distribution function that... we cannot found
in the spreadhsheet softwares like Microsoft Excel. To be able to get the closed form of the
inverse Cauchy CDF we start from the CDF proven previously:
P(X ≤ x) =
1
π
arctan(x) +
1
2
(4.7.595)
and therefore if we let:
y =
1
π
arctan(x) +
1
2
(4.7.596)
We immediately get the inverse CDF:
x = tan π y −
1
2
(4.7.597)
That is useful in finance as we know (see section Numerical Methods) that to simulate a Cauchy
variable when the use the inverse transforme sampling:
X = tan π U[0,1] −
1
2
(4.7.598)
4.7.6.16 Beta Distribution
Let us first recall that the Euler Gamma function is defined by the relation (see section Differ-
ential And Integral Calculus):
Γ(z) =
+∞ˆ
0
e−x
xz−1
dx (4.7.599)
We proved (see section Differential And Integral Calculus) that a non-trivial property of this
function is:
Γ(z + 1) = zΓ(z) (4.7.600)
Let us now write:
Γ(a)Γ(b) = lim
R→+∞
¨
AR
e−x−y
xa−1
yb−1
dxdy (4.7.601)
where:
AR = {(x, y)|x ≥ 0, y ≥ 0, x + y ≤ R} (4.7.602)
By the change of variables:
x
y
=
u − v
v
(4.7.603)

Draft
we get:
Γ(a)Γ(b) = lim
R→+∞
¨
AR
e−x−y
xa−1
yb−1
dxdy = lim
R→+∞
Rˆ
0
e−u


uˆ
0
(u − v)a−1
vb−1
dv

 du
(4.7.604)
For the internal integral we now use the substitution v = ut, 0 ≤ t ≤ 1 and therefore we find:
Γ(a)Γ(b) = lim
R→+∞
Rˆ
0
e−u


uˆ
0
(u − v)a−1
vb−1
dv

 du
= lim
R→+∞
Rˆ
0
e−u
ua+b−1
du
1ˆ
0
(1 − t)a−1
tb−1
dt
déf
= B(a, b)
+∞ˆ
0
e−u
ua+b−1
du
= B(a, b)Γ(a + b)
(4.7.605)
The function B that appears in the expression above is named beta function and therefore we
have:
B(a, b) =
Γ(a)Γ(b)
Γ(a + b) (4.7.606)
Now that we have defined what we name the beta function, consider the two parameters a
0, b 0 and consider also the special relation below as the beta distribution or beta law
(there are several formulations of the beta distribution and a very important one is studied in
detail in the section of Quantitative Management):
Pa,b(x) =
xa−1
(1 − x)b−1
B(a, b)
1]0,1[ (4.7.607)
where:
B(a, b) =
1ˆ
0
xa−1
(1 − x)b−1
dx (4.7.608)
We first check that Pa,b(x) that is effectively a distribution function (without getting into too
much details ...)
+∞ˆ
−∞
Pa,b(x)x =
+∞ˆ
−∞
xa−1
(1 − x)b−1
B(a, b)
· 1]0,1[dx =
+∞ˆ
−∞
xa−1
(1 − x)b−1
B(a, b)
· 1]0,1[dx
=
1
B(a, b)
+∞ˆ
−∞
xa−1
(1 − x)b−1
dx =
B(a, b)
B(a, b)
= 1
(4.7.609)
Let us now calculate the expected mean:
µ =
+∞ˆ
−∞
xPa,b(x)dx =
1
B(a, b)
1ˆ
0
xa
(1 − x)b−1
dx
=
B(a + 1, b)
B(a, b)
=
Γ(a + 1)Γ(b)
Γ(a + b + 1)
·
Γ(a + b)
Γ(a)Γ(b)
=
a
a + b
(4.7.610)

Draft
by using the relation:
Γ(z + 1) = zΓ(z) (4.7.611)
and its variance:
σ2
=
+∞ˆ
−∞
(x − µ)2
f(x)x =
1
B(a, b)
1ˆ
0
(x − µ)2
xa−1
(1 − x)b−1
dx
=
1
B(a, b)


1ˆ
0
xa+1
(1 − x)b−1
x − 2µ
1ˆ
0
xa
(1 − x)b−1
dx + µ2
1ˆ
0
xa−1
(1 − x)b−1
dx


=
B(a + 2, b) − 2µ2
B(a, b) + µ2
B(a + 1, b)
B(a, b)
=
B(a + 2, b) − µ2
B(a, b)
B(a, b)
(4.7.612)
As we know that:
Γ(z + 1) = zΓ(z) and B(a, b) =
Γ(a)Γ(b)
Γ(a + b)
(4.7.613)
we ﬁnd:
B(a + 2, b) =
µ · (a + 1)
a + b + 1
(4.7.614)
and therefore:
σ2
=
µ · (a + 1)
a + b + 1
− µ2
=
ab
(a + b)2(a + b + 1)
(4.7.615)
Examples of plots of the beta distribution function for (a, b) = (0.1, 0.5) in red, (a, b) =
(0.3, 0.5) in green, (a, b) = (0.5, 0.5) in black, (a, b) = (0.8, 0.8) in blue, (a, b) = (1, 1)
in magenta, (a, b) = (1, 1.5) in cyan, (a, b) = (1, 2) in gray, (a, b) = (1.5, 2) in turquoise,
(a, b) = (2, 2) in yellow,(a, b) = (3, 3) in gold color:
Figure 4.93 – Some Beta law mass functions

Draft
Here is a plot example of the beta distribution and cumulative distribution for the parameters
(a, b) = (2, 3):
Figure 4.94 – Beta law (mass and cumulative distribution function)
4.7.6.17 Gamma Distribution
The Euler Gamma function being known, consider two parameters a 0, λ 0 and let us
define the Gamma distribution (or Gamma law) as given by the relation (density function):
Pa,λ(x) =
xa−1
e−λx
+∞ˆ
0
xa−1
e−λx
dx
· 1]0,+∞[
(4.7.616)
By the change of variables t = λx we obtain:
+∞ˆ
0
xa−1
e−λx
dx =
Γ(a)
λa
(4.7.617)
and we can then write the relation in a more conventional form that we find frequently in the
literature:
Pa,λ(x) =
xa−1
e−λx
Γ(a)
λa
· 1]0,+∞[ (4.7.618)
and it is under this notation that we find this distribution function in Microsoft Excel 11.8346
under the name qewwehsƒ„@ A and its inverse by qewwesx†@ A.
Let us now see a simple property of the Gamma distribution that will be partially useful for the
study of the Welch statistical test . First recall that we have shown above that:

Draft
∀y ∈ R fY (y) =
1
cte
fX
y
cte
(4.7.619)
Let us write Y = cte
X, then we have immediately:
fY (y) =
ya−1
e
λ
y
cte
Γ(a)
λ
cte
1]0,+∞[ (4.7.620)
So the multiplication by a constant of random variable that follows a Gamma distribution has
only for effect of dividing the parameter λ by the same constant. This is the reason why λ is
named NewTermscale parameter.
If a ∈ N, the Gamma distribution at the denominator becomes (see section Differential And
Integral Calculus) the factorial (a − 1)!. The Gamma function can then be written:
Pa,λ =
xa−1
λa
e−λx
(a − 1)!
=
(xλ)a−1
(a − 1)!
λe−λx
(4.7.621)
This particular notation of the Gamma distribution is named the Erlang distribution that we
ﬁnd naturally in the theory of queues and that is very important in practice!
Remark
If a = 1 then Γ(a) = 1 and xa−1
= 1 and we fall back on the exponential distribution.
Then we check with a similar reasoning to this of the beta distribution that Pa,λ(x) is a distribu-
tion function:
+∞ˆ
−∞
Pa,λdx = 1 (4.7.622)
Examples of plots of the beta distribution function for (a, λ) = (0.5, 1) in red, (a, λ) = (1, 1)
in green, (a, λ) = (2, 1) in black, (a, λ) = (4, 2) in blue, (a, λ) = (16, 8) in magenta:

Draft
Figure 4.95 – Some Gamma law mass functions
and a plot example of the Gamma distribution and cumulative distribution for the parameters
(a, γ) = (4, 1):
Figure 4.96 – Gamma law (mass and cumulative distribution function)
The Gamma function has also for expected mean:
µ =
+∞ˆ
−∞
xf(x)dx =
λa
Γ(a)
+∞ˆ
−∞
xa
e−λx
dx =
λa
Γ(a)
Γ(a + 1)
λa+1
λa
Γ(a)
aΓ(a)
λa+1
=
a
λ
(4.7.623)

Draft
and for variance:
σ2
=
+∞ˆ
−∞
(x − µ)2
f(x)dx =
λa
Γ(a)
+∞ˆ
−∞
(x − µ)2
xa−1
e−λx
dx
=
λa
Γ(a)


+∞ˆ
−∞
xa+1
e−λx
x − 2µ
+∞ˆ
−∞
xa
e−λx
x + µ2
+∞ˆ
−∞
xa−1
e−λx
dx


=
λa
Γ(a)
Γ(a + 2)
λa+2
+ µ2 Γ(a)
λa
− 2µ
Γ(a + 1)
λa+1
=
1
λ2Γ(a)
Γ(a + 2) + a2
Γ(a) − 2aΓ(a + 1)
=
1
λ2Γ(a)
(a + 1)aΓ(a) + a2
Γ(a) − 2a2
Γ(a) =
a
λ2
(4.7.624)
Let us now prove a property of the Gamma distribution that will permit us later in this chapter,
during our study of the analysis of variance and conﬁdence intervals based on small samples,
another extremely important property of the Chi-square distribution.
As we know, the distribution function of a random variable following a Gamma function of
parameters a, λ 0 is:
Pa,γ(x) = f(x) =
e−λx
xa−1
Γ(a)
λa
1[0,+∞] (4.7.625)
with (see section Differential And Integral Calculus) the Euler Gamma function:
Γ(a) =
+∞ˆ
0
e−x
xa−1
dx (4.7.626)
Moreover, when a random variable follows a Gamma function we often notice it in the following
way:
X + Y = γ(a, λ) (4.7.627)
Let X, Y be two independent variables. We will prove that if X = γ(p, λ) and Y =
gamma(q, λ), hence with the same scale parameter, then:
X + Y = γ(p + q, λ) (4.7.628)
We write f the density function of the pair X, Y, f(x) the density function of X and fY the
density function of Y . Because X and Y are independent, we have:
f(x, y) = fX(x) · fY (y) (4.7.629)
for all x, y 0.
Let Z = X + Y . The distribution function of Z is therefore:
F(z) = P(Z z) = P(X + Y z) =
¨
D
f(x, y)xy (4.7.630)

Draft
where D = {(x, y)|x + y ≤ z}.
Remark
As we already know we name such a calculation a convolution and statisticians often
have to handle such entities because they work on many random variables that they have
to sum or even to multiply.
Simplifying:
F(z) =
+∞ˆ
−∞
z−xˆ
−∞
We perform the following change of variable x = x, y = s − x. The Jacobian is therefore (see
Differential And Integral Calculus):
J =
∂x
∂x
∂x
∂s
∂y
∂x
∂y
∂s
=
1 0
−1 0
= 1 (4.7.632)
Therefore with the new integration limits s = x + y = x + (z − x) = z we have:
F(z) =
+∞ˆ
−∞
zˆ
−∞
fX(x)fY (s − x)dsdx =
zˆ
−∞
+∞ˆ
−∞
fX(x)fY (s − x)dxds (4.7.633)
If we denote by g the density function Z we have:
F(z) =
zˆ
−∞
+∞ˆ
−∞
fX(x)fY (s − x)dxds =
zˆ
−∞
g(s)ds (4.7.634)
Then it follows:
g(s) =
+∞ˆ
−∞
fX(x)fY (s − x)dx (4.7.635)
fX and fY being null when the argument is negative, we can change the limits of integration:
g(s) =
sˆ
0
fX(x)fY (s − x)dx for s 0 (4.7.636)
Let us calculate g:
g(s) =
λp+q
e−λs
Γ(p)Γ(q)
sˆ
0
xp−1
(s − x)q−1
dx (4.7.637)

Draft
After the change of variable x = st we obtain:
g(s) =
λp+q
e−λs
Γ(p)Γ(q)
sp+q−1
1ˆ
0
tp−1
(1 − t)q−1
dt =
λp+q
e−λs
Γ(p)Γ(q)
sp+q−1
B(p, q) (4.7.638)
where B is the beta function we saw earlier in our study of the beta distribution. But we have
also proved the relation:
B(p, q) =
Γ(p)Γ(q)
Γ(p + q)
(4.7.639)
Therefore:
B(p, q) =
λp+q
e−λs
Γ(p + q)
sp+q−1
(4.7.640)
More explicitly:
g(x, y) =
λp+q
e−λ(x+y)
Γ(p + q)
(x + y)p+q−1
(4.7.641)
Which finally gives us:
P(Z z) = P(X + Y z) =
sˆ
0
g(s)ds =
sˆ
0
λp+q
e−λs
Γ(p + q)
sp+q−1
ds (4.7.642)
This shows that that if two random variables follow a Gamma distribution then their sum will
also follow a Gamma distribution with parameters:
X + Y = γ(p + q, λ) (4.7.643)
So the Gamma distribution is stable by addition as are all distribution arising from the Gamma
distribution that we will see below.
4.7.6.18 Chi-Square (pearson) Distribution
The chi-square distribution (also called chi-square law or Pearson law) has a very impor-
tant place in the industrial practice for some common hypothesis tests (see far below...) and is
by definition only a particular case of the Gamma distribution in the case where a = k/2 and
λ = 1/2, when k is a positive integer:
Pk(x) =
1
2
k
2 Γ(k/2)
x
k
2
−1
e−x
2 · 1[0,+∞[ (4.7.644)
This relation that connects the chi-square distribution with the Gamma distribution is important
in the in Microsoft Excel 11.8346 as the function grshsƒ„@ A returns the confidence level and
not the distribution function. Then you must use the function qewwehsƒ„@A with the parameters
given above (except that you must take the inverse of 1/2: also 2 as parameter) to get the
distribution and cumulative functions.
The reader who wishes to check that the Chi-square distribution is only a special case of the
Gamma distribution can write in Microsoft Excel 14.0.6123:

Draft
agrsƒFhsƒ„@PBxDPBkD„‚…iA
aqewweFhsƒ„@xDkDID„‚…iA
All calculations made previously still apply and we get immediately:
µ = k, σ2
= 2k (4.7.645)
Examples of plots of the chi-2 distribution function for k = 1 in red, k = 3 in green, in black,
k = 4 in blue:
Figure 4.97 – Some Chi-2 χ2 law mass functions
and a plot example of the chi-2 distribution and cumulative distribution for the parameter k = 2:
Figure 4.98 – Chi-2 χ2 law (mass and cumulative distribution function)

Draft
In the literature, it is traditional to write:
X = χ2
k or X = χ2
(k) (4.7.646)
to indicate that the distribution of the random variable X is a chi-square distribution. Further-
more it is common to name the parameter k degree of freedom and abbreviate it df.
The χ2
distribution is therefore a special case of the Gamma distribution and by taking k = 2
we also find the exponential distribution (see above) for λ = 1/2:
P2(x) =
1
21Γ(1)
x2/2−1
e−x/2
· 1[0,+∞[ =
1
2
e
−
x
2 1[0,+∞[ (4.7.647)
Moreover, since (see section Differential And Integral Calculus):
Γ
1
2
=
√
π (4.7.648)
the χ2
distribution with k equal to unity can be written as:
P1(x) =
1
21/2Γ(1/2)
x1/2−1
e−x/2
· 1[0,+∞[ =
1
√
2πx
e−x/2
· 1[0,+∞[ (4.7.649)
Finally, let us finish with a fairly large property in the field of statistical tests that we will
investigate a little further and particularly for confidence intervals of rare events. Indeed, the
reader can check in a spreadsheet software like Microsoft Excel 14.0.6123 that we have:
a€ysƒƒyxFhsƒ„@ x ∈ N, µ,TRUE)
aIEgrsƒFhsƒ„@ 2µ; 2(x + 1),TRUE)
aIEqewweFhsƒ„@ 2µ, x + 1,TRUE)
So we need to prove this relation between law χ2
and Poisson distributions. See it starting from
the Gamma distribution:
Pa,λ(x) =
xa−1
e−λx
Γ(a)
λa
1]0,+∞[ (4.7.650)
If we write λ = 1/2 and a = k/2 then we have the χ2
distribution with k degrees of freedom:
Pk(x) =
1
2k/2Γ(k/2)
xk/2−1
e−x/2
1]0,+∞[ (4.7.651)
Now remember that we have seen in the section Sequences And Series, the following Taylor
(Maclaurin) serie from order n − 1 around 0 to λ with integral rest:
eλ
=
n−1
k=0
λk
k!
ex
(0) +
λˆ
0
en
(t)
(λ − 1)n−1
(n − 1)!
dt =
n−1
k=0
λk
k!
+
λˆ
0
et (λ − 1)n−1
(n − 1)!
dt (4.7.652)
=
u=λ−t
n−1
k=0
λk
k!
−
0ˆ
λ
eλ−u un−1
(n − 1)!
du =
n−1
k=0
λk
k!
+
λˆ
0
eλ−u un−1
(n − 1)!
du (4.7.653)

Draft
We multiply by e−λ
:
eλ
e−λ
= e−λ


n−1
k=0
λk
k!
+
λˆ
0
eλ−u un−1
(n − 1)!
du

 ⇒ 1 =
n−1
k=0
λk
k!
e−λ
+
λˆ
0
e−u un−1
(n − 1)!
du
(4.7.654)
And therefore:
n−1
k=0
λk
k!
e−λ
= 1 −
λˆ
0
e−u un−1
(n − 1)!
du (4.7.655)
Now, let us focus on the term:
λˆ
0
e−u un−1
(n − 1)!
du (4.7.656)
and make a ﬁrst change of variable:
λˆ
0
e−u un−1
(n − 1)!
du =
u=x/2
λˆ
0
1
2n−1(n − 1)!
xn−1
e−x/2 1
2
dx =
λˆ
0
1
2n(n − 1)!
xn−1
e−x/2
dx
(4.7.657)
and a second change of variable (caution! the k in the change of variable is not the same as this
in the Poisson sum...):
λˆ
0
1
2n(n − 1)!
xn−1
e−x/2
dx =
n=k/2
λˆ
0
1
2k/2
k
2
− 1 !
xk/2−1
e−x/2
dx (4.7.658)
However, we have shown in the section of Differential And Integral Calculus that if x is a
positive integer:
x! = Γ(x + 1)! (4.7.659)
Then it comes:
k
2
− 1 ! = Γ
k
2
− 1 + 1 = Γ
k
2
(4.7.660)
Finally we have:
λˆ
0
1
2k/2
k
2
!
xk/2−1
e−x/2
dx (4.7.661)
where we ﬁnd out the chi-2 distribution under the integral! So at the end:
n−1
k=0
λk
k!
e−λ
= 1 −
λˆ
0
1
2k/2
k
2
!
xk/2−1
e−x/2
dx (4.7.662)
This explains the formulas given above for the spreadsheet software.

Draft
4.7.6.19 Student Distribution
The Student distribution (or Student’s law) of parameter k is defined by the relation:
P(x) =
Γ
k + 1
2
Γ
k
2
√
kπ
1 +
x2
k
−k+1
2
(4.7.663)
with k being the degree of freedom of the χ2
distribution underlying the construction of the
Student function as we will see.
Let us indicate that this distribution can also be obtained in Microsoft Excel 11.8346 using the
„hsƒ„@ A function and its inverse by „sx†@A.
It is indeed a distribution function because it also satisfies (remains to be proved directly, but as
we will see it is the product of two distribution functions thus indirectly...):
+∞ˆ
−∞
Pk(x)x = 1 (4.7.664)
Let us see the easiest proof to justify the provenance of the Student distribution and that will
also be very useful further in statistical inference and analysis of variance.
f(x) =
1
√
2π
e−x2/2
(4.7.665)
1. If X and Y are two independent random variables with respective densities fX, fY , the
distribution of the pair (X, Y ) has a density f satisfying (axiom of probabilities!)
f(x, y) = fX(x)fY (y) (4.7.666)
2. The distribution N(0, 1) is given by (see above):
f(x) =
1
√
2π
e−x2/2
(4.7.667)
3. The distribution χ2
n is given by (see above):
f(y) =
1
2n/2Γ(n/2)
e−y/2
y
n
2
−1
(4.7.668)
for and y ≥ 0 and n ≥ 1.
4. The function Γ is defined for all α 0 by (see section Differential and Integral Calculus):
Γ(α) =
+∞ˆ
0
e−x
xα−1
dx (4.7.669)

Draft
and satisﬁes (see section Differential and Integral Calculus):
Γ(α − 1) =
Γ(α)
α − 1
(4.7.670)
for α ≥ 2.
These reminders made, now consider a random variable X that follows the distribution N(0, 1)
and Y a random variable following the distribution χ2
n.
We assume X and Y being independent and we consider the random variable (this is at the
origin the historical study of the Student distribution in the framework of statistical inference
which led to deﬁne this variable for which we will deepen the origin later):
T =
√
n
X
√
Y
=
X
Y/n
(4.7.671)
We will prove that T follows a Student distribution of parameter n.
Proof 4.27.4. Let F and f and be respectively the repartition and density functions and T,
fX, fY the density functions of X, Y and (X, Y ) respectively. Then we have for all t ∈ R:
F(t) = P(T t) = P
√
n
X
√
Y
t =
¨
D
f(x, y)dxdy =
¨
D
fX(x)fY (y)dxdy
(4.7.672)
where:
D = (x, y) ∈ R × R∗
|x ≤
t
√
y
√
n
(4.7.673)
the imposed positive and non-zero value and y being due to the fact that it is under a root and
furthermore at the denominator. Thus:
F(t) =
¨
D
fX(x)fY (y)dxdy =
+∞ˆ
0
fY (y)dy
t
√
y/
√
nˆ
−∞
fX(x)dx =
+∞ˆ
0
fY (y)Φ(t
√
y/
√
n)dy
(4.7.674)
where because X follows a Normal N(0, 1) distribution.
Φ(u) =
uˆ
−∞
1
√
2π
e−x2/2
dx (4.7.675)
is the Normal centered reduced cumulative distribution. Thus, we obtain the density distribution
function of T by deriving F:
f(t) = F (t) =
1
√
n
+∞ˆ
0
fY (y)fX(t
√
y/
√
n)
√
ydy (4.7.676)

Draft
because (the derivative of a function is equal to its derivatives multiplied by its inner derivative):
dΦ(t
√
n/
√
t)
dt
= fX(t
√
n/
√
t)
√
y
√
n
(4.7.677)
Therefore:
f(t) =
1
√
n
+∞ˆ
0
fY (y)fX(t
√
y/
√
n)
√
ydy
=
1
√
n
+∞ˆ
0
1
2n/2Γ(n/2)
e−y/2
yn/2−1 1
√
2π
e−(t
√
y/
√
n)2/2√
ydy
=
1
√
n
1
2n/2Γ(n/2)
1
√
2π
+∞ˆ
0
e−y/2
yn/2−1
e−(t
√
y/
√
n)2/2√
ydy
=
1
2n/2Γ(n/2)
√
2πn
+∞ˆ
0
e−
y(1+t2/n)
2 yn/2−1
dy
(4.7.678)
By making the change of variable:
u =
y
2
1 +
t2
n
⇒



dy =



2
1 +
t2
n


 du
dy = 2
1+t2
n
u
(4.7.679)
(4.7.680)
we get:
f(t) =
1
2n/2Γ(n/2)
√
2πn
2
1 + t2/n
(n+1)/2
+∞ˆ
0
e−u
u(n−1)/2
du
=Γ


n + 1
2


=
Γ
n + 1
2
2n/2Γ(n/2)
√
2π
√
n
2
1 + t2/n
(n+1)/2
=
Γ n+1
2
Γ(n/2)
√
πn
1 +
t2
n
−n+1
2
(4.7.681)
what is the Student distribution of parameter n.
Q.E.D.

Draft
Let us now prove what is the mean of the Student distribution:
T =
√
n
X
√
Y
(4.7.682)
We have:
E(T) = E(
√
nX)E
1
√
Y
(4.7.683)
But E
1
√
Y
exists if and only if n ≥ 2. Effectively for n = 1:
E
1
√
Y
=
+∞ˆ
0
fY (y)
√
y
dy =
1
2
n
2 Γ(n/2)
+∞ˆ
0
e−y
2 y
n−3
2 dy (4.7.684)
and:
+∞ˆ
0
e−y
2 y
1−3
2 dy =
+∞ˆ
0
e−y
2
y
dy
1ˆ
0
e−y
2
y
dy e−1
2
+∞ˆ
0
1
y
dy → +∞ (4.7.685)
Whereas for n ≥ 2 we have:
+∞ˆ
0
e−y
2 y
n−3
2 dy = 2
n−1
2
+∞ˆ
0
e−u
u
n−3
2 du = 2
n−1
2 Γ
n − 1
2
+∞ (4.7.686)
Thus, for n = 1 the mean does not exist.
So for n ≥ 2:
E(T) =
√
n E(X)
=0
E
1
√
Y
= 0 (4.7.687)
Now let us see the value of the variance. So we have:
V(T) = E(T2
) − E(T)2
(4.7.688)
First we will discuss the existence of E(T2
). We have trivially:
E(T2
) = nE
X2
Y
= nE(X2
)E
1
Y
(4.7.689)
X follows a Normal centered reduced distribution thus:
V(X) = 1 = E(X2
) − E(X)2
=0
= E(X2
) ⇒ E(X2
) = 1 (4.7.690)
With regard to E
1
Y
we have:
E
1
Y
=
+∞ˆ
0
fY (y)
y
dy =
1
2
n
2 Γ(n/2)
+∞ˆ
0
e−y
2 y
n
2
−2
dy =
1
2
n
2 Γ(n/2)
+∞ˆ
0
e−u
u
n
2
−2
du =
Γ n
2
− 1
2Γ(n/2)
(4.7.691)

Draft
where we made the change of variable u = y/2. But the integral defining Γ
n
2
− 1 converges
only if n ≥ 3. Therefore E(T2
) exists if and only if n ≥ 3 so it’s value is according to
the properties of the Euler Gamma function demonstrated in the chapter of Differential And
Integral Calculus:
nE (T) = n
Γ n
2
− 1
2Γ(n/2)
=
n
n − 2
(4.7.692)
Therefore for n ≥ 3:
V(T) =
n
n − 2
(4.7.693)
It is also important to note that this law is symmetrical about 0!
Plot example of the Student distribution and cumulative distribution for the parameter k = 3:
Figure 4.99 – Student T law (mass and cumulative distribution function)
4.7.6.20 Fisher Distribution
The Fisher distribution (or Fisher-Snedecor distribution) of parameters k and l is defined by
the relation:
P(x) =
Γ
k + 1
2
k
l
k
2
Γ
k
2
Γ
l
2
x
k
2
−1
1 +
kx
l
−k+l
2
(4.7.694)
if x ≥ 0. The parameters k and l are positive integers and correspond to the two degrees of
freedom of the underlying chi-square distributions. This distribution is often denoted by Fk,l or
by F(k, l) and can be obtained in Microsoft Excel 11.8346 with phsƒ„@ A distribution. It is
indeed a distribution function because it satisfies the property:
+∞ˆ
0
Fk,l(x)dx = 1 (4.7.695)

Draft
Let us see the easiest proof to justify the provenance of the Fisher distribution and that we will
be us also very useful further in statistical inference and analysis of variance.
For this proof, recall that:
1. The distribution χ2
n is given by (see above):
f(y) =
1
2n/2Γ(n/2)
e−y/2
yn/2−1
(4.7.696)
for y ≥ 0 and n ≥ 1.
2. The Euler Gamma function Γ is deﬁned for all α 0 by (see section Differential and
Integral Calculus):
Γ(α) =
+∞ˆ
0
e−x
xα−1
dx (4.7.697)
Let X, Y be two independent random variables following respectively the distributions
χ2
n and χ2
m.
We consider the random variable:
T =
X/n
Y/m
(4.7.698)
We will prove that the distribution of T is the Fisher-Snedecor distribution of parameters n, m.
Let us note for this purpose F and f the distribution and cumulative distribution function of T
and fX, fY , f density functions of X, Y and respectively (X, Y ). We have for all t ∈ R:
F(t) = P(T t) = P
X/n
Y/m
t =
¨
D
f(x, y)dxdy =
¨
D
where:
D = (x, y) ∈ R∗
× R∗
|x ≤
nt
m
y (4.7.700)
where the imposed positive values comes in fact that behind them there is a chi-square for x and
y. Therefore:
F(t) =
+∞ˆ
0
fY (y)dy
nty
mˆ
0
fX(x)dy (4.7.701)
We obtain the density function of T by deriving F. First the inner derivative:
f(t) = F (t) =
n
m
+∞ˆ
0
fY (y)fX
nty
m
y ydy (4.7.702)

Draft
Then explicitly because:
f(y) =
1
2m/2Γ(m/2)
e−y/2
ym/2−1
and f
nt
m
y =
1
2n/2Γ(n/2)
e− nt
2m
y nt
m
y
n/2−1
(4.7.703)
we then have:
f(t) =
n
m
+∞ˆ
0
1
2m/2Γ(m/2)
e−y/2
y
m
2
−1 1
2
n
2 Γ(n/2)
e− nt
2m
y nt
m
y
n
2
−1
ydy
=
1
2n/2Γ(n/2)2m/2Γ(m/2)
n
m
+∞ˆ
0
e−y/2
ym/2−1
e− nt
2m
y nt
m
y
n
2
−1
ydy
=
1
2n/2Γ(n/2)2m/2Γ(m/2)
n
m
nt
m
n/2−1
+∞ˆ
0
e−y/2
ym/2−1
e− nt
2m
y
yn/2−1
ydy
=
1
2
n+m
2 Γ(n/2)Γ(m/2)
n
m
nt
m
n
2
−1
+∞ˆ
0
y
n+m
2
−1
e−y
2 (1+nt
m )dy
(4.7.704)
By making the change of variable:
u =
y
2
1 +
nt
m
⇒ y =
2u
1 + nt
m
⇒ dy = du
2
1 + nt
m
(4.7.705)
we get:
f(t) =
n
m
1
2
n+m
2 Γ(n/2)Γ(m/2)
nt
m
n
2
−1
2
1 + nt
m
n+m
2
+∞ˆ
0
u
n+m
2
−1
e−u
du
Γ(n+m
2 )
=
n
m
Γ n+m
2
Γ(n/2)Γ(m/2)
nt
m
n
2
−1
1 +
nt
m
−n+m
2
=
n
m
n
2
Γ n+m
2
Γ(n/2)Γ(m/2)
t
n
2
−1
1 +
nt
m
−n+m
2
(4.7.706)

Draft
4.7.6.21 Benford Distribution
This distribution was discovered first in 1881 by Simon Newcomb, an American astronomer,
after he saw that the wear (and so the use) of the preferred first pages of logarithms tables (at
this time there we compiled into books). Frank Benford, around 1938 remarked at his turn this
unequal wear, believing he was the first to formulate this law that unduly bears his name today
and arrived at the same results after having listed tens of thousands of data (lengths of rivers ,
stock quotes, etc.).
There is also one possible explanation: we need more often to extract the logarithm of numbers
starting with 1 that numbers starting with 9, implying that the first are in bigger quantity than
the second one.
Although this idea may seem to him quite implausible, Benford began to test his hypothesis.
Nothing more simple: he study tables of numerical values and calculates the percentage of
occurrence of the left-most digit (first decimal). The results obtained confirm his intuition:
From these data, Benford found experimentally that the cumulative probability of a number
First position number 1 2 3 4 5 6 7 8 9
Apparition probability (%) 30.1 17.6 12.5 9.7 7.9 6.7 5.8 5.1 4.6
Table 4.27 – Occurrence of a digit following the Benford distribution
beginning with the digit n (except 0) is (we will prove this later) is given by the relation:
P(n) = log10 1 +
1
n
(4.7.707)
named Benford distribution (or Benford law).
Here is a Maple plot of the previous function:
Figure 4.100 – Plot of the Benford function (cumulative distribution function)

Draft
It should be noted that this distribution applies only to lists of values that are natural, that is to
say numbers with physical meaning. It obviously does not work on a list of numbers randomly
drawn.
The Benford distribution has been tested on all kinds of tables: length of the rivers of the world,
country area, election results, price list of grocery store ... It is true almost every time.
The distribution is said to to be independent of the selected unit. If we take for example a
supermarket price list , it also works well with the costs expressed in dollars as with the same
costs converted into Euros.
This strange phenomenon remained unexplained and little studied until quite recently. Then a
general proof was given in 1996, which uses the central limit theorem.
As surprising as it may seem, this distribution has found application: it is said that the IRS use
it to detect false statements. The principle is based on the restriction seen above: Benford’s
distributions applies only to values with physical meaning.
Thus, if there is a universal probability distribution P(n) on such numbers, they should be
invariant under scaling such that:
P(kn) = f(k)P(n) (4.7.708)
If:
ˆ
P(n)dn = 1 (4.7.709)
Then:
ˆ
P(kn)dn =
1
k
(4.7.710)
and the normalization of the distribution gives:
f(k) =
1
k
(4.7.711)
If we derivate P(kn) = f(k)P(n) with respect to k we obtain:
d
dk
P(kn) =
d
dk
1
k
P(n) ⇒ nP (kn) = P(n)
d
dk
1
k
⇒ nP (kn) = −P(n)
1
k2
(4.7.712)
choosing k = 1 we have:
nP (n) = −P(n) (4.7.713)
This differential equation has for solution:
P(n) =
1
n
(4.7.714)
This function is not strictly speaking a distribution fonction (it diverges) and secondly, the
physics and human laws impose limits.

Draft
So we have to compare this distribution with respect to an arbitrary reference. Thus, if the
decimal number studied contains power of 10 (10 in total: 0, 1, 2, 3, 4, 5, 6, 7, 9) the probability
that the ﬁrst nonzero digit (decimal) is D is also given by:
PD =
D+1ˆ
D
P(n)dn
1ˆ
1
0P(n)dn
(4.7.715)
The limits of the integral are from 1 to 10 because the null value is prohibited.
The integral in the denominator gives:
1ˆ
0
0P(n)dn =
1ˆ
0
0
1
n
dn = ln(10) − ln(1) = ln(10) (4.7.716)
The integral in the numerator gives:
D+1ˆ
D
P(n)dn =
D+1ˆ
D
1
n
dn = ln(D + 1) − ln(D) = ln
D + 1
D
(4.7.717)
Finally:
PD =
ln
D + 1
D
ln(10)
=
ln 1 +
1
D
ln(10)
(4.7.718)
By the properties of logarithms (see section Functional Analysis) we have:
PD = log10 1 +
1
D (4.7.719)
However, the Benford’s distribution applies not only to non-scaling data but also to numbers
from any sources. Explain this case involves a more rigorous investigation using the central
limit theorem. This demonstration was conducted only in 1996 by T. Hill by an approach using
the distribution of distributions.
To summarize an important part of everything we’ve seen so far, the picture below is very
useful because it summarizes the relation between 76 most common univariate distributions 76
(57 continuous and 19 discrete):

Draft
Figure 4.101 – Relations between distributions (Source: AMS Lawrence M. Leemis and Jacquelyn T. McQueston)
4.7.7 Likelihood Estimators
What follows is of extreme importance in the ﬁeld of statistics and is used widely in practice.
It is important therefore to pay attention! Besides the fact that we will use this technique in
this chapter, we shall ﬁnd it in the chapter of Numerical Methods for advanced and generalized
linear regression and also in the chapter of Industrial Engineering in the context of parametric
estimation of reliability.
We assume that we have observations x1, x2, x3, ..., xn which are realizations of unbiased
independent random variables (in the sense that they are randomly selected from a batch)
X1, X2, X3, ..., Xn of a unknown probability distribution but having the same one.
Suppose we proceed by trial and error to estimate the unknown probability distribution P. One
way to proceed is to ask if the observations x1, x2, x3, ..., xn had a high probability to get out or
not with this arbitrary probability distribution P.
We need for this to calculate the joint probability that the observations x1, x2, x3, ..., xn had to
get out with the probabilities p1, p2, p3, ..., pn. This joint probability is equal to (see section

Draft
Probabilities):
n
i=1
P(Xi = xi) (4.7.720)
noting by the letter P the assumed probability distribution associated to p1, p2, p3, ..., pn. You
must admit that it would be particularly awkward, at the intuition level of risk, to choose a
probability distribution (with its parameters!) that minimizes this quantity...
Instead, we will seek the probabilities p1, p2, p3, ..., pn (or the associated parameters of the prob-
ability distribution) that maximizes n
i=1 P(Xi = xi), that is to say, that makes the observations
x1, x2, x3, ..., xn the most likely possible.
This leads us to seek the parameter(s) θ that maximizes the quantity:
Ln(θ) =
n
i=1
Pθ(Xi = xi) (4.7.721)
and where the parameter θ is often in undergraduate school level problems a first order moment
(mean) or second order moment (variance).
The quantity L is named likelihood. It is a function of the parameter(s) θ and observations θ.
The value(s) of the parameter(s) θ that maximize the likelihood Ln(θ) are called maximum
likelihood estimators (MLE estimators).
In the very special case but useful of the Normal distribution, one of the parameters θ will
be the variance (see a little further concrete example) and can be considered intuitive to the
physicist that to maximize the probability, the standard deviation should be as small as possible
(so that the maximum numbers of events are in the same interval). Thus, when we calculate an
MLE which is the smallest among several possible, then we are talking about a UMV estimator
for Uniform Minimum Variance Unbiased because their own variance should be as small as
possible. This can be demonstrated (but the proof is not very elegant) using the definition of
the Fisher Information and the Fréchet theorem (or Rao-Cramer) that makes use of the Cauchy-
Schwartz inequality (see section Vector Calculus) and the analogy between mean and scalar
product ... This demonstration will not be on this website.
Let us still do five small examples (very classic, useful and important in the industry) with in
order of importance (i.e. not necessarily in order of ease...) the distribution function of Gauss-
Laplace (Normal distribution), the Poisson distribution, the binomial distribution (and so the
Geometric distribution), the Weibull distribution and finally the Gamma distribution.
Remark
These five examples are important as used in SPC (statistical process control) in various
international companies around the world (see section Industrial Engineering).

Draft
4.7.7.1 Normal Distribution MLE
Let be x1, x2, ..., xn an n-sample of identically distributed random variables assumed to follow
a Gaussian-Laplace (Normal) distribution of parameters µ and σ2
.
We are looking what are the values of the maximum likelihood estimators θ that maximize the
likelihood Ln(θ) of the Normal distribution?
Remark
It is trivial that the maximum likelihood estimators vector is here:
θ = (µ, σ) (4.7.722)
We have prove earlier above that the density of a Gaussian random variable was given by:
P(x, µ, µ) =
1
σ
√
2π
e
−
(x − µ)2
2σ2
(4.7.723)
The likelihood is then given by:
L(µ, σ) =
n
i=1
P(xi, µ, µ) =
1
σn
√
2π
n e
−
1
2σ2
n
i=1
(xi − µ)2
(4.7.724)
Maximize a function or maximize its logarithm is equivalent therefore the log-likelihood will
be:
ln(L(µ, σ)) = −
n
2
ln(2π) − n ln(σ) −
1
2σ2
n
i=1
(xi − µ)2
(4.7.725)
To determine the two estimators of the Normal distribution, first let us fix the standard deviation.
To do this, we derive ln(L(µ, σ)) over µ and look for what the average value of the function is
equal to zero.
It remains after simplification the following term that is equal to zero:
n
i=1
(xi − µ)2
(4.7.726)
Thus, the maximum likelihood estimator of the expected mean of the Normal distribution is
after rearrangement:
¯µ =
1
n
n
i=1
xi (4.7.727)
and we see that it is simply the arithmetic mean (or also named sample mean).

Draft
Let us now fix the mean. The cancellation ln(L(µ, sigma)) of the derivative over σ leads us to:
∂
∂σ
ln(L(µ, σ)) = −
n
σ2
−
1
σ3
n
i=1
(xi − µ)2
= 0 (4.7.728)
This allows us to write the maximum likelihood estimator for the standard deviation (the vari-
ance when the mean is known under the an assumed distribution also supposed known!):
ˆσ =
1
n
n
i=1
(xi − µ)2
(4.7.729)
that some people also name Pearson standard deviation...
Even if it is a little bit redundant some people as us to show the proof of the estimator of the
covariance matrix (and therefore the correlation matrix).
Remember that we have prove earlier that for the bivariate case we have:
f(X1, X2) =
1
2π|Σ|1/2
e
−
1
2
−−−−→
(x−µ)T Σ−1
−−−−→
(x−µ)
(4.7.730)
In fact the relation is the same for the multivariate case with T!
The log-likelihood is therefore immediate by analogy with the univariate case:
ln(L(µ, Σ) = −
nT
2
ln(2π) −
T
2
ln(Σ) −
1
2
T
i=1
(xi − µ)T
Σ−1
(xi − µ) (4.7.731)
That we can also write (as Σ is diagonal and Σ−1
also):
ln(L(µ, Σ) = −
nT
2
ln(2π) −
T
2
ln(Σ) −
1
2
T
i=1
tr (xi − µ)T
Σ−1
(xi − µ)
= −
nT
2
ln(2π) −
T
2
ln(Σ) −
1
2
T
i=1
tr Σ−1
(xi − µ)T
(xi − µ)
:= −
nT
2
ln(2π) −
T
2
ln(Σ) −
1
2
tr Σ−1
S
(4.7.732)
Where by definition and using the estimator of the mean:
S =
T
i=1
(xi − ˆµ)T
(xi − ˆµ) (4.7.733)
Then we deduce that:
∂
∂Σ−1
ln(L(ˆµ, Σ) =
T
2
Σ −
1
2
Σ = 0 (4.7.734)
and we get finally:
ˆΣ =
1
T
S (4.7.735)
However, we have not yet defined what is a good estimator! What we mean here is:

Draft
• If the mean of an estimator is equal to itself, we say that this estimator is unbiased and
that’s obviously what we want!
• If the mean of an estimator is not equal to itself, then we say that this estimator is biased
and is necessarily less good...
In the previous example, the average is unbiased (this is trivial as the average of the arithmetic
mean is equal to itself). But what about the variance (verbatim the standard deviation)?
A simple little calculation by linearity of the mean (since the random variables are identically
distributed) will give us the answer in the case where the theoretical average (mean) is approxi-
mated as in practice (industry) by the estimator of the mean (most common case).
So we have for the calculation of the mean of the sample variance:
E(ˆσ2
) = E(V(X)) = E
1
n
n
i=1
(xi − µ)2
= E
1
n
n
i=1
(x2
i − 2xi ˆµ + ˆµ2
)
= E
1
n
n
i=1
x2
i − 2
1
n
n
i=1
xi ˆµ +
1
n
n
i=1
ˆµ2
) = E
1
n
n
i=1
x2
i − 2ˆµ
1
n
n
i=1
xi +
1
n
nˆµ2
)
= E
1
n
n
i=1
x2
i − 2ˆµ2
+ ˆµ2
) = E
1
n
n
i=1
x2
i − 2ˆµ) =
1
n
n
i=1
E(x2
i ) − E(ˆµ2
)
(4.7.736)
However, as the variables are supposed to be identically distributed:
E(ˆσ2
) =
1
n
n
i=1
E(x2
i ) − E(ˆµ2
) =
1
n
n
i=1
E(x2
) − E(ˆµ2
) = E(x2
) − E(ˆµ2
) (4.7.737)
And as we have (Huyghens theorem):
V(X) = E(X2
) − E(X2
) (4.7.738)
V(ˆµ) = E(ˆµ2
) − E(ˆµ)2
= E(ˆµ2
) − E(X)2
(4.7.739)
wherein the second relation can be written only because we use the maximum likelihood esti-
mator of the average (empirical average). Therefore combining the two above relations with the
prior-previous one we get:
E(ˆσ2
) = E(x2
) − E(ˆµ2
) = V(X) + E(X)2
− V(ˆµ) + E(X)2
= V(X) − V(ˆµ) (4.7.740)
and as:
V(X) = σ2
and V(ˆµ) =
σ2
n
(4.7.741)
Finally we have:
E(ˆσ2
) = σ2
−
σ2
n
= 1 −
1
n
σ2
=
n − 1
n
σ2
(4.7.742)

Draft
so we have a bias of at least one standard error:
−
σ2
n
(4.7.743)
then we say that this estimator has a negative bias (it underestimates the true value!).
We also note that the estimator tends towards to an unbiased estimator of the variance (USV)
when the number of items tends to infinity n → +∞. We say that we have a asymptotically
unbiased or asymptotically unbiased estimator.
It is important to note that we have yet proved that the empirical variance tends towards the
theoretical variance when n tends to infinity and ... that the data follows or not a Normal
distribution!
Remark
An estimator is named consistent estimator if it converges in probability, when n →
+∞, towards the true parameter value.
By the properties of the mean, we get:
E(E(ˆσ2
)) = E(ˆσ2
) = E
n − 1
n
σ2
(4.7.744)
We have then:
σ =
n
n − 1
ˆσ2 =
n
n − 1
1
n
n
i=1
(xi − ˆµ)2 =
1
n − 1
n
i=1
(xi − ˆµ)2 (4.7.745)
simply called the standard deviation ... (that must not be confused with the standard error
as we shall see later).
So we finally summarize as following the two important previous results:
1. The biased maximum likelihood estimator or also named empirical standard deviation
or sample standard deviation or Pearson standard deviation ... is therefore given by:
ˆσ =
1
n
n
i=1
(xi − ˆµ)2
(4.7.746)
when n → +∞. We find this standard deviation depending on the context (by tradition)
noted in five other ways that are:
σ∗
, S∗
, σ∗, S∗, Sn (4.7.747)
and sometimes (but this is very awkward because it often generates confusion with the
unbiased estimator) σ or S.

Draft
2. The unbiased maximum likelihood estimator or simply named standard deviation:
σ =
1
n − 1
n
i=1
(xi − ˆµ)2
(4.7.748)
which as we can see is a consistent estimator (when n tends to infinity it tends to the
biased maximum likelihood estimator).
We find this standard deviation depending on the context (by tradition) noted in three
other ways that are:
σ, S, Sn−1 (4.7.749)
We find these last two notations often in tables and in many softwares and we will use them
later in the development of confidence intervals and hypothesis testing!
For example, in the Microsoft Excel 11.8346 the unbiased estimator is given by the ƒ„hi†@ A
function and the non-biased by ƒ„hi†€@A.
In total, this make us is three estimators for the same indicator! As in the overwhelming majority
of cases of the industry the mean is not known, we usually use the last two relations bordered
above. Now this is where comes the vicious part: when we calculate the bias of this two
estimators, the first is biased, the second is not. So we tend to use only the latter. Nay! Because
we could also talk about the variance and precision of an estimator, which are also important
criteria for judging the quality of an estimator relative to another. If we were to calculate the
variance of the two estimators, then the first, which is biased, is smaller than the second which
is unbiased variance! All that to say that the criteria of bias is not (by far) the only one to be
study to judge the quality of an estimator.
Finally, it is important to remember that the factor −1 in the denominator of the unbiased
maximum likelihood estimator stems from the need to correct the mean of the biased estimator
initially subtracted by one time the standard error!
4.7.7.2 Poisson Distribution MLE
Using the same method as for the Normal (Gauss-Laplace) distribution, we will seek the maxi-
mum likelihood estimators of the Poisson distribution which for recall is given by:
Pk =
µk
k!
e−µ
(4.7.750)
Thus, the likelihood is given by
L(µ) =
n
i=1
P(xi, µ) =
µ
n
i=1
xi
n
i=1 xi!
e−µn
(4.7.751)

Draft
Maximize a function or maximize its logarithm is equivalent therefore:
ln[L(µ)] = ln(µ)
n
i=1
xi −
n
i=1
ln(xi!) − µn (4.7.752)
We are now looking to maximize it:
∂ ln[L(µ)]
∂µ
=
1
µ
n
i=1
xi − n = 0 (4.7.753)
and thus we obtain the only maximum likelihood estimator that will be:
ˆµ =
1
n
n
i=1
xi (4.7.754)
It is quite normal to ﬁnd in this example the sample mean because it is the best possible estimator
for the parameter of the Poisson distribution (which also represents the mean of a Poisson
distribution).
Knowing that the standard deviation of this particular distribution (see above during the devel-
opment of the Poisson distribution) is the square root of the mean, then we have for the standard
deviation maximum likelihood :
ˆσ =
1
n
i=1 xi
(4.7.755)
Remark
We show in the same way identical results for the exponential distribution that is widely
used in preventive maintenance and reliability!
4.7.7.3 Binomial (and Geometric) Distribution MLE
Using the same method as for the Normal distribution (Gauss-Laplace) and the Poisson dis-
tribution, we will seek the maximum likelihood estimator of the Binomial which we recall, is
given by:
P(N, k) = CN
k · P =
N!
k!(N − k!)
pk
qN−k
=
N!
k!(N − k!)
pk
(1 − p)N−k
(4.7.756)
Accordingly, the likelihood is given by:
L(p) =
N
i=1
P(xi, p) = CN
k pk
(1 − p)N−k
(4.7.757)
It should be remembered that the factor following the combinatorial term already expressed the
successive variables according to what we saw during our study of the Bernoulli and Binomial
distribution functions. Hence the disappearance of the product in the preceding equality.

Draft
ln[L(p)] = ln(CN
k ) + k ln(p) + (N − k) ln(1 − p) (4.7.758)
We are now looking to maximize it:
∂ ln[L(p)]
∂p
=
k
p
−
N − k
1 − p
= 0 (4.7.759)
The reader may have perhaps noticed that the binomial coefficient has disappeared. There-
fore, we immediately deduce that the estimator of the binomial distribution is the same as the
geometric distribution.
Which gives:
k(1 − p) − p(N − k) = k − kp − pN + pk = k − pN = 0 (4.7.760)
from which we derive the maximum likelihood estimator:
ˆp =
k
N
(4.7.761)
This result is quite intuitive if we consider the classic example of a coin that has a chance on
tow of dropping on one of its faces. The probability p being the number of times k a given face
where was observed in the total number of tests (all sides combined).
Remark
In practice, it is not as easy to apply these estimators! We must carefully consider which
are most suitable for a given experiment and ideally also calculate the mean squared error
(standard error) of each of the estimators of the mean (as we have already done for the
empirical mean earlier). In short... it is a long process of reflection.
4.7.7.4 Weibull Distribution MLE
We saw in the section of Industrial Engineering a very detailed study of the three-parameter
Weibull distribution with its standard deviation and mean because as we mentioned it is quite
used in the field of reliability engineering.
Unfortunately, the three parameters of this distribution are unknown in practice. Using estima-
tors however we can determine the expression of two of the three assuming γ as zero. This
gives us the following Weibull distribution named Weibull distribution with two parameters:
P(x, β, η) =
β
η
x
η
β−1
e
−
x
n
β
(4.7.762)
and for recall with β 0 and η 0.

Draft
L(β, η) =
n
i=1
P(xi, β, η) =
n
i=1
β
η
xi
η
β−1
e
−
xi
n
β
=
β
η
n n
i=1
xi
η
β−1
e
−
n
i=1
xi
n
β
(4.7.763)
ln(L(β, η)) = ln





β
η
n n
i=1
xi
η
β−1
e
−
n
i=1
xi
η
β





= n ln
β
η
−
1
ηβ
n
i=1
xβ
i +
n
i=1
ln
xi
η
β−1
= n ln
β
η
− η−
β
n
i=1
xβ
i + (β − 1)
n
i=1
ln
xi
η
(4.7.764)
Now we seek to maximize this by remembering that (see section Differential And Integral
Calculus):
d
dx
ax
= ax
ln(a) and
d
dx
a−x
= −ax
ln(a) (4.7.765)
then:
∂ ln(β, η)
∂β
= n
1
β
+
1
etaβ
ln(η)
n
i=1
xβ
i −
1
ηβ
n
i=1
xβ
i ln(xi) +
n
i=1
ln
xi
η
= n
1
β
+
1
ηβ
n
i=1
xβ
i (ln(η) − ln(xi)) +
n
i=1
ln
xi
η
= n
1
β
−
1
ηβ
n
i=1
xβ
i (ln(xi) − ln(η)) +
n
i=1
ln
xi
η
= n
1
β
−
1
ηβ
n
i=1
xβ
i ln
xi
η
+
n
i=1
ln
xi
η
= 0
(4.7.766)
And we get for the second parameter:
∂ ln(β, η)
∂η
= −n
1
ν
−
β
ηβη
n
i=1
xβ
i + (1 − β)n
1
η
= 0 (4.7.767)
then:
−
1
ηβ
n
i=1
xβ
i − n = 0 (4.7.768)
Finally to resume with the correct notations (and in the resolution order in practice):
n
1
ˆβ
−
1
ˆηˆβ
n
i=1
x
ˆβ
i ln
xi
ˆη
+
n
i=1
ln
xi
ˆη
= 0 and −
1
ˆηˆβ
n
i=1
x
ˆβ
i − n = 0 (4.7.769)

Draft
Solving these equations involves heavy computations and we can a priori do nothing with that
in conventional spreadsheets softwares such as Microsoft Excel or Open Ofﬁce Calc without
programming (at least as far as we know...).
We then take a different approach by writing our Weibull distribution with two parameters as
follows:
P(x, β, θ) =
β
θ
xβ−1
e
xβ
θ (4.7.770)
with for recall β 0 and θ 0.
Therefore the likelihood is given by:
L(β, η) =
n
i=1
P(xi, β, θ) =
n
i=1
β
θ
xβ−1
i e
−
xβ
i
θ =
β
θ
n
e
−
n
i=1
xβ
i
θ
n
i=1
xβ−1
i (4.7.771)
ln(L(β, η)) = ln





β
θ
n
e
−
n
i=1
xβ
i
θ
n
i=1
xβ−1
i





= n ln
β
θ
−
1
θ
n
i=1
xβ
i +
n
i=1
ln(xβ−1
i ) = n ln
β
θ
−
1
θ
n
i=1
xβ
i + (β − 1)
n
i=1
ln(xi)
(4.7.772)
Now we seek to maximize this by remembering that (see section Differential And Integral
Calculus):
d
dx
ax
= ax
ln(a) and
d
dx
a−x
= −ax
ln(a) (4.7.773)
then:
∂ ln(β, θ)
∂β
=
n
β
−
1
θ
n
i=1
xβ
i ln(xi) +
n
i=1
ln(xi) = 0 (4.7.774)
And we have for the second parameter:
∂ ln(β, θ)
∂θ
= −
n
θ
+
1
θ2
n
i=1
xβ
i = 0 (4.7.775)
It is then immediate that:
¯θ =
1
n
n
i=1
xβ
i (4.7.776)

Draft
injected into the equation:
n
β
−
1
θ
n
i=1
xβ
i ln(xi) +
n
i=1
ln(xi) = 0 (4.7.777)
We get:
n
β
−
n
i=1
xβ
i ln(xi)
1
n
n
i=1
xβ
i
+
n
i=1
ln(xi) = 0 (4.7.778)
simplifying:
n
i=1
xβ
i ln(xi)
1
n
n
i=1
xβ
i
−
1
β
=
1
n
n
i=1
ln(xi) (4.7.779)
The resolution of the two equations (in order from top to bottom):
n
i=1
xβ
i ln(xi)
1
n
n
i=1
xβ
i
−
1
β
−
1
n
n
i=1
ln(xi) = 0
1
n
n
i=1
xβ
i − ¯θ
(4.7.780)
can easily be calculated with the Target Tool of Microsoft Excel or Open Office Calc.
4.7.7.5 Gamma Distribution MLE
Here we will use a technique named method of moments to determine the estimators of the
parameters of the Gamma distribution.
Suppose that X1, ..., Xn are independent and identically distributed random variables according
to the Gamma distribution with density:
Pa,λ(x) =
xa−1
e−λx
Γ(a)
λa
1[0,+∞[ (4.7.781)
We seek to estimate a, λ. For this, we first determine some theoretical moments. The first
moment is the expected mean that as we have proved before is given by:
E(X) = m1 =
a
λ
(4.7.782)

Draft
and the second moment, the mean of the square of the random variable, is as we have implicitly
proved in the proof of the variance of the Gamma distribution given by:
E(X2
) = m2 =
a(a + 1)
λ2
(4.7.783)
We then express the relationship between the parameters and the theoretical moments:
m1 =
a
λ
m2 =
a(a + 1)
λ2
(4.7.784)
The resolution of this simple system gives:
a =
m2
1
m2 − m2
1
λ =
m2 − m2
1
m2
1
(4.7.785)
Once this system established, the method of moments consist to use the empirical moments, i.e.
for our example the ﬁrst two, ˆm1, ˆm2:
ˆm1 =
X1 + ... + Xn
n
ˆm2 =
X2
1 + ... + X2
n
n
(4.7.786)
that we deﬁne as equal to the true theoretical moments ... Therefore, it comes:
a =
ˆm2
1
ˆm2 − ˆm2
1
λ =
ˆm2 − ˆm2
1
ˆm2
1
(4.7.787)
4.7.8 Finite Population Correction Factor
Now we prove another result which we will be required in some statistical tests that we will see
later.
Suppose we have a population of N individuals that we we represent by the set {1, 2, ..., N} and
a random variable X which is an application of {1, 2, ..., N} in R. We denote by xi = X(i).
The mean of X is thus given by:
µ =
1
N
N
i=1
xi (4.7.788)

Draft
Remember the variance of X is by definition:
σ =
1
N
N
i=1
(xi − µ)2
(4.7.789)
Now we consider the set E of samples of size n taken in {1, 2, ..., N} with 0 n N. Each
individual has a probability of being drawn equal to:
1
N
1
N − 1
...
1
N − (n − 1)
=
(N − n)!
N!
(4.7.790)
We are interested in the random variable ¯X defined on E and that equal to the sample mean.
More specifically:
¯X(i1, ..., in) =
1
n
n
k=1
xik
(4.7.791)
To calculate the variance V( ¯X), we will ¯X express as a sum of random variables. Indeed, if we
define the variables Xk with k = 1...N by:
Xk(i1, ..., in) =
xk if k ∈ {i1, ..., in}
0 otherwise
(4.7.792)
(4.7.793)
We have naturally by the previous definition (see with caution the sum limits!):
¯X =
1
n
N
k=1
Xk (4.7.794)
and thus we get:
V( ¯X) =
σ
n
=
1
n
N
k=1
V(Xk) +
N
i=j
cov(Xi, Xj) (4.7.795)
The random variables Xk are not independent in pairs, in fact as we shall see, their covariances
are not zero if N is finite. Otherwise (zero covariance), we find a result already proved earlier:
V( ¯X) = σ ¯X =
σ2
n
(4.7.796)
So we need to calculate the variances V(X) and covariances cov(Xi, Xj).
For this purpose we will use the Huyghens relation and we will start by calculating the mean
E(Xk):
E(Xk) = P(Xk = xk)xk (4.7.797)
But P(Xk = xk) is the probability that a sample contains k. This probability is obviously equal
to n/N and therefore:
E(Xk) = P(Xk = xk)xk =
n
N
xk (4.7.798)

Draft
Similarly we obtain:
E(X2
k ) = P(Xk = xk)xk =
n
N
x2
k (4.7.799)
We can therefore calculate the variance V(Xk):
V(Xk) = E(X2
k ) − E(Xk)2
=
n
N
x2
k −
n
N
xk
2
=
n(N − n)
N2
x2
k (4.7.800)
To calculate the covariances we need now to calculate the means E(XiXj):
E(XiXj) = P(Xi = xi, Xj = xj)xixj (4.7.801)
But P(Xi = xi, Xj = xj) is the probability that a sample contains i and j. This probability is
obviously given by:
n
N
n − 1
N − 1
(4.7.802)
and therefore:
E(XiXj) =
n(n − 1)
N(N − 1)
xixj (4.7.803)
We can now compute the covariance:
cov(Xi, Xj) = E(XiXj) − E(Xi)E(Xj) =
n(n − 1)
N(N − 1)
xixj −
n2
N2
xixj = −
n(N − n)
N2(N − 1)
xixj
(4.7.804)
We are now able to simplify V( ¯X):
V( ¯X) =
1
n
N
k=1
V(Xk) +
N
i=j
cov(Xi, Xj) =
1
n
n(N − n)
N2
N
k=1
x2
k −
n(N − n)
N2(N − 1)
N
i=j
xixj
(4.7.805)
Using Huyghens theorem we get:
V(Xk) = σ2
= E(X2
k ) − E(Xk)2
= E(X2
k ) − µ2
= E(X2
k ) − µ2
⇔ σ2
+ µ2
= E(X2
k )
(4.7.806)
Using the result proved above and previous relation:
E(X2
k ) =
n
N
x2
k = σ2
+ µ2
(4.7.807)
Therefore:
x2
k =
N
n
σ2
+
N
n
µ2
⇔
N
k=1
x2
k =
N2
n
σ2
+ µ2
(4.7.808)

Draft
For the double sum
N
i=j
xixj, we have:
N
i=j
xixj =
N
i=1
xi(x1 + ... + xi−1 + xi+1 + ... + xN ) =
N
i=1
xi(Nµ − xi)
= Nµ
N
i=1
xi −
N
i=1
x2
i = NµNµ −
N
i=1
x2
i = N2
µ2
−
N
i=1
x2
i
= N2
µ2
−
N2
n
σ2
+ µ2
(4.7.809)
Therefore:
V( ¯X) =
1
n
n(N − n)
N2
N2
n
σ2
+ µ2
−
n(N − n)
N2(N − 1)
N2
µ2
−
N2
n
σ2
+ µ2
=
1
n
(N − n) σ2
+ µ2
−
n(N − n)
N − 1
µ2
−
1
n
σ2
+ µ2
=
1
n
(N − n) σ2
+ µ2
−
n(N − n)
N − 1
µ2
+
N − n
N − 1
σ2
+
N − n
N − 1
µ2
=
1
n
(N − n)σ2
+
N − n
N − 1
σ2
+ µ2 N − n
N − 1
−
n(N − n)
N − 1
+ (N − n)
=
1
n
(N − n)σ2
+
N − n
N − 1
σ2
+ µ2 −nN + n2
+ N2
− nN
N − 1
=
1
n
(N − n)σ2
+
N − n
N − 1
σ2
+ µ2 (N − n)2
N − 1
=
1
n
N(N − n)
N − 1
σ2
+ µ2 (N − n)2
N − 1
= ..???.. = σ2 N − n
n(N − 1)
(4.7.810)
Thus:
σ ¯X = V( ¯X) =
σ
√
n
N − n
N − 1
(4.7.811)
The famous factor:
fcp =
N − n
N − 1
≤ 1 (4.7.812)
that we have already encountered during our study the hypergeometric distribution is named
finite correction factor (on finite population) and has the effect of reducing the standard error
especially as n is large.
4.7.9 Confidence Intervals
Until now we have always determined the likelihood estimators or simple estimators (variance,
standard deviation) from theoretical statistics distributions or measured on an entire population
of data.

Draft
Definition: A confidence interval is a pair of numbers that defines (a posteriori) the range
of possible values with a certain cumulative probability of an (punctual) estimator of a given
statistical indicator form a sample of an experience (the range of the statistical indicator being
usually calculated using real measured parameters). It is the most common statistical case.
We now turn to the task that consists naturally to ask ourselves what must be the sample sizes
of our measured data to have some validity (C.I.: confidence interval) for our estimators or even
to which confidence interval correspond a given standard deviation or quantile in a Normal cen-
tered reduced distribution (for large samples), in a chi-square distribution, Student distribution
or Fisher distribution (we will see the last two cases of small sample sizes in the section on
analysis of variance or ANOVA) when the man or variance are known or unknown respectively
on all or part of the given population.
It is important to know that these confidence intervals often use the central limit theorem that
will be proved late (to avoid any possible frustration) and the developments that we will do now
are also useful in the field of (a posteriori) Hypotheses Tests that have a major role in statistics
and therefore indirectly in all fields of science!!!
Finally, it could be useful to indicate that a large numbers of organizations (private or institu-
tional) make false statistics because the assumptions and conditions of use of these confidence
intervals (verbatim hypotheses tests) are not rigorously verified or simply omitted or worse, the
whole base (measurements) is not collected in the rules of art (reliability of the data collection
and reproductibility protocols not validated by scientific peer).
The reader must also know that we have put many other confidence interval techniques detailed
proofs related for example for regression techniques in the section of Theoretical Computing.
Remark
The practitioner should be very careful about the calculation of confidence intervals and
the use of hypothesis testing in practice. This is why, to avoid trivial usage error or
interpretation, it is important to refer to the following international standards eg: ISO
2602:1980 •, ISO 2854:1976 (Statistical interpretation of data - Techniques of estimation
and tests relating to means and variances), ISO 3301:1975 (Statistical interpretation of
data - Comparison of two means in the case of paired observations), ISO 3494:1976 (Sta-
tistical interpretation of data - Effectiveness of tests relating to means and variances ), ISO
5479:1997 (Statistical interpretation of data - Tests for departure from the normal distri-
bution ), ISO 10725:2000 + ISO 11648-1:2003 + ISO 11648-2:2001 (Sampling plans
and procedures for acceptance for control of bulk materials), ISO 11453:1996 (Statisti-
cal interpretation of data - Tests and confidence intervals relating to proportions), ISO
16269-4:2010 (Statistical interpretation of data - Detection and treatment of outliers),
ISO 16269-6:2005 (Statistical interpretation of data - Determination of statistical toler-
ance intervals), ISO 16269-8:2004 (Statistical Interpretation of data - Determination of
prediction intervals), ISO / TR 18532:2009 (Guidelines for the application of statistical
quality and industrial standards ).

Draft
4.7.9.1 C.I. on the Mean with known Variance
Let’s start with the simplest and most common case that is the determination of the number of
individuals to have some confidence in the average of the measurements of a random variable
assumed to follow a Normal distribution.
First let us recall that we showed at the beginning of this chapter that the standard error (standard
deviation of the mean) was under the assumptions of independent and identically distributed
variables (i.i.d.):
σ ¯X =
σ
√
n
(4.7.813)
Now, before we go any further, consider X as a random variable following a Normal distribution
with mean µ and standard deviation σ. We would like that the random variable has for example
95% cumulative probability of being in a given bounded symmetric interval. Which is therefore
expressed as follows:
P(µ − δ X µ + δ) = 0.95 (4.7.814)
Remark
Therefore with a confidence interval of 95% you will be right a posteriori 19 times out
of 20, or any other level of confidence or risk level α (1-confidence level, 5%) that you
will be set up in advance. On average, your conclusions will therefore be good, but we
can never know whether a particular decision is good! If the risk level is very low but
the event still occurs, specialists then speak about a large deviation or a black swan.
Management of outliers is addressed in ISO 16269-4:2010 Detection and treatment of
outliers that any engineer doing business statistics has to follow.
By centering and reducing the random variable:
P −
δ
σ
X − µ
σ
δ
σ
= 0.95 (4.7.815)
Let us now write Y the reduced centered variable:
P −
δ
σ
Y − P Y
δ
σ
= 0.95 (4.7.816)
Since the Normal centered reduced distribution is symmetric:
1 − 2P Y
δ
σ
= 0.95 (4.7.817)
Therefore:
P Y
δ
σ
= 0.025 (4.7.818)
From there reading statistical tables of the standard Normal distribution (or by using a simple
spreadsheet software), we have to satisfy the equality that:
δ
σ
∼= 1.96 ⇔ δ ∼= 1.96σ (4.7.819)
Which can easily be obtained with Microsoft Excel 11.8346 by using the function:

Draft
aExy‚wƒsx†@@IEHFWSAGPA
As noted in the traditional way in the general case other than the 95% one (Z is the random
variable corresponding to the half quantile of the chosen threshold of the standard Normal
distribution):
δ ∼= Zσ (4.7.820)
Now, consider that the variable X on which we wish to make statistical inference is the av-
erage (and we show later that it follows a Normal distribution centered reduced distribution).
Therefore:
δ ∼= Zσ = Z
σ
√
n
(4.7.821)
Then we get:
n ∼=
Z2
σ2
δ2
(4.7.822)
from which we obviously take (normally...) the upper integer value... The latter notation is
usually written in the following way highlighting better the width of the conﬁdence interval of
an underlying threshold level:
n ∼=
Zα/2σ
δ
2
(4.7.823)
Relation named sample size estimation by Normal distribution.
Thus, we now know the number of individuals we must have to ensure to get a given precision
interval δ (margin of error) around the mean and that for a given percentage measures are in
this range and assuming the theoretical standard deviation σ is known (or imposed) in advance
(typically used in quality engineering or surveys).
In other words, we can calculate the number n of individuals to measure to ensure a given con-
ﬁdence interval (associated to the quantile Z) of the measured average assuming the theoretical
standard deviation known (or imposed) and wishing a precision δ in absolute value of the mean.
However ... in reality, the variable Z comes from the central limit theorem (see below) that
gives for a large sample size (approximately):
Z = Zn =
Mn − µ
σ/
√
n
=
¯X − µ
σ/
√
n
(4.7.824)
Rearranging we get the:
µ = ¯X − Z
σ
√
n
(4.7.825)
and as Z can be negative or positive then it is more logic to write this as:
µ = ¯X ± Z
σ
√
n
(4.7.826)

Draft
Thus:
¯X − Z
σ
√
n
µ ¯X + Z
σ
√
n
(4.7.827)
That engineers sometimes write:
LCL µ UCL (4.7.828)
where LCL is the lower confidence limit and UCL the upper confidence limit. This is the Six
Sigma terminology (see section Industrial Engineering).
And we have seen earlier that for a confidence interval of 95% we have Z = 1.96. And since
the Normal distribution is symmetric:
95% = 1 − 5% = 1 − α = 1 − 2 × 2.5% = 1 − 2 × 0.025 (4.7.829)
Thus we finally write the one sample Z test:
¯X − Zα/2
σ
√
n
µ ¯X + Zα/2
σ
√
n (4.7.830)
As we have already mentioned, and we will prove a little further, the arithmetic reduced centered
mean of a series of independent and identically distributed random variables with finite variance
asymptotically follows a standard Normal distribution, this is why the confidence interval above
is very general! This is why we sometimes speak of asymptotic confidence interval of the
mean.
These intervals obviously have for origin the fact that we work very often in statistics with
samples and not the entire available population. The selected sampling thus affects the value of
the punctual estimator. We then speak of sampling fluctuation.
In the particular case of an IC (confidence interval) at 95%, the last relation will be written:
¯X − Z0.025
σ
√
n
µ ¯X + Z0.025
σ
√
n
(4.7.831)
Sometimes we find the prior-previous inequality in the following equivalent notation:
¯X − Zα/2σ ¯X µ ¯X + Zα/2σ ¯X (4.7.832)
or more rarely with the following general notation (for all intervals):
¯X − ME µ ¯X + ME (4.7.833)
where ME stands for margin of error. We are thus now able to estimate population sizes
needed to obtain a certain level of confidence α in an outcome or to estimate the confidence
interval in which is the theoretical mean knowing the experimental (empirical) average and
the estimator maximum likelihood of the standard deviation. We can of course therefore also
determine the a posteriori probability that the mean is outside a given range ... (one as the other
being widely used in the industry).

Draft
Finally, note that from the previous result, we deduce immediately the stability property of the
Normal distribution (shown above) the following test that we find in many statistical softwares:
Zcalc.
=
( ¯X2 − ¯X1) − (µ2 − µ1)
σ2
1
n
+
σ2
2
n
(4.7.834)
named bilateral Z test on the difference of two means or also sometimes called two sample
Z test a with the corresponding confidence interval:
( ¯X2 − ¯X1) − Zα/2
σ2
1
n
+
σ2
2
n
≤ µ2 − µ1 ≤ ( ¯X2 − ¯X1) + Zα/2
σ2
1
n
+
σ2
2
n
(4.7.835)
And this is not because two averages are significantly different than their confidence intervals
do not overlap! As shows the graph below obtained with Minitab 16 software where the test-Z
of the difference is significant at 95%:
Figure 4.102 – Line plot illustration of the overlay of two confidence with 95% confidence interval
while their mean is significantly different to a confidence level of 95%.
Remark
The size of the parent population for the relations developed above does not come into
consideration in the calculations of confidence intervals or even not in the sample size,
and because it is considered as infinite. So be careful not sometimes not to have sample
sizes that are larger than the actual parent population...

Draft
4.7.9.2 C.I. on the Variance with known Mean
Let’s start by demonstrating a fundamental property of the Chi-square distribution:
If a random variable X follows a Normal centered reduced distribution X = N( , ∞) then its
square follows a chi-square distribution of 1 degree of freedom:
X2
= χ2
(1) (4.7.836)
This result is sometimes named a Wald statistics and any statistical test using it directly (we
should better speak about a test family) can be designer under the name Wald’s test (for a
concrete example see Cochran-Mantel-Haenszel test in the section of Numerical Methods).
Proof 4.27.5. To prove this property, it sufﬁces to calculate the density of the random X2
variable with X = N( , ∞). However, if X = N( , ∞) and if we set Y = X2
, then for all
y ≥ 0 we get:
P(Y y) = P(X2
y) = P(−
√
y Y
√
y) (4.7.837)
Since the standard Normal distribution is symmetric about 0 for the random variable X, we can
write:
P(Y y) = 2P(0 X
√
y) (4.7.838)
Denoting by Φ the cumulative distribution function of the standard normal distribution, we
have:
P(Y y) = 2Φ(
√
y) − 2 × 0.5 = 2Φ(
√
y) − 1 (4.7.839)
and as:
Φ(0) = P(N(0, 1) ≤ 0) =
1
√
2π
0ˆ
−∞
e−k/2
dk = 0.5 (4.7.840)
therefore:
P(Y ≤ y) = 2Φ(
√
y) − 2 · 0.5 = 2Φ(
√
y) − 1 (4.7.841)
The cumulative distribution function of the random variable Y = X2
is thus given by:
P(Y ≤ y) = 2Φ(
√
y) − 1 (4.7.842)
if y is greater than or equal to zero, null if y is less than zero. We will note this cumulative
distribution fY (y) for the further calculations.
Since the density distribution function is the derivative of the cumulative distribution function
and X follows a Normal centered reduced distribution so we reduced for the random variable
X:
P(X = x) = Φ (x) =
d
dx
Φ(x) =
1
√
2π
e−x2/2
(4.7.843)

Draft
and then it follows for the probability distribution of Y (which is the square of X for reminder!):
d
dy
fY (y) =
d
dy
(2Φ(
√
y) − 1) = 2Φ (
√
y)
d
dy
√
y =
1
√
y
Φ (
√
y) =
1
√
y
1
√
2π
e−y/2 1
√
2πy
e−y/2
(4.7.844)
this last expression corresponds is exactly the relation we obtained during our study of the
chi-square distribution imposing a degree of freedom equal to the unit.
The theorem is therefore proved, that is if X follows a Normal centered reduced distribution
while its square follows a Chi-square distribution of 1 degree of freedom as:
XN(0,1)
2
= χ2
(1) (4.7.845)
Q.E.D.
This type of relation is used in industrial processes and their control (see section Industrial
Engineering).
We will now use a result proved during our study of the Gamma distribution. We have effec-
tively seen that the sum of two random variables following a Gamma distribution also follows
a Gamma distribution where the two parameters are added:
X + Y = γ(p + q, λ) (4.7.846)
As the Chi-square distribution is a special case of the Gamma distribution, the same result
applies.
To be more precise, this is equivalent to say: If X1, ..., Xkare random independent and identi-
cally distributed (i.i.d.) variables N(0, 1) then by extension of the above proof where we have
shown that:
XN(0,1)
2
= χ2
(1) (4.7.847)
and by the property of linearity of the Gamma distribution, then sum of their squares follows a
chi-square distribution of degree k such that:
χ2
(k) = X2
1 + X2
2 + . . . + X2
k (4.7.848)
Thus, the distribution of χ2
of k degrees freedom is the probability distribution of the sum of
squares of k Normal centered reduced variables linearly independent of each other. It is in
fact the linearity property of the chi-square distribution (implicitly the linearity of the Gamma
distribution)!
Now see another signiﬁcant property of the chi-square distribution: If X1, ..., Xn are indepen-
dent and identically distributed N(µ, σ) (thus the same mean and the same standard and follow-
ing a Normal distribution) random variables and if we write the maximum likelihood estimator

Draft
variance by:
S2
∗ =
1
n
n
i=1
(Xi − µ)2
(4.7.849)
then, the ratio of the random variable S2
∗ on the standard deviation assumed to be known for the
entire population (the true standard deviation or theoretical standard deviation!) multiplied
by the number of individuals n population follows a chi-square distribution of degree n such
that:
χ2
(n) = χ2
n =
nS2
∗
σ2
(4.7.850)
This result is named the Cochran theorem or Fisher-Cochran theorem (in the particular
case of Gaussian samples) and thus gives us a distribution for the empirical standard deviations
(whose parent law is a Normal distribution!).
Using the value of the standard deviation proved during our study of chi-square distribution we
have:
V
nS2
∗
σ2
= V(χ2
(n)) = 2n (4.7.851)
But n and σ are imposed and are therefore considered as constants. We have therefore:
n2
σ4
V(S2
∗) = V(χ2
(n)) = 2n ⇒ V(S2
∗) = σS2
∗
= σ4 2
n
(4.7.852)
And therefore we have an expression of the standard deviation of the empirical standard devia-
tion if we know the standard deviation of the population:
σS∗ = σ2 2
n
(4.7.853)
But we have prove during our study of estimators that:
ˆσ =
n − 1
n
σ (4.7.854)
It follows:
ˆσS∗ =
n − 1
n
σS2
∗
=
n − 1
n
σ2 2
n
=
2(n − 1)
n2
σ2
(4.7.855)
It follows therefore the sometimes important relation in the practice of the estimator of the
standard deviation of ... the standard deviation:
ˆσS∗ =
2(n − 1)
n2
σ2
(4.7.856)
Recall that the parent population is said to be inﬁnite if the sample selection with replacement
or if the size N of the parent population is much higher than this of the sample of size n.

Draft
Remarks
R1. In the laboratory, the X1, ..., Xn can be seen as a class of individuals of the same
product identically studied by different research teams with instruments of the same
precision (standard deviation of the measure identically equal).
R2. S∗2 is the interclass variance also called explained variance. So it gives a measure
of the variance occurring in different laboratories.
What is interesting here is that from the calculation of the chi-square distribution and by know-
ing n and the standard deviation σ2
it is possible to estimate the interclass variance (and also
interclass standard deviation).
To see that this latter property is a generalization of the basic relation:
χ2
(n) = X1
1 + X2
2 + ... + X2
n (4.7.857)
it sufﬁces to see that the random variable nS2
∗/σ2
is a sum of n squares of N(0, 1) independent
of each other. Indeed, recall that a centered reduced random variable (see our study of the
Normal distribution) is given by:
Y =
X − µ
σ
(4.7.858)
Therefore:
nS2
∗
σ2
=
n
σ2
1
n
n
i=1
(Xi − µ)2
=
n
i=1
Xi − µ
σ
2
(4.7.859)
However, since the random variables X1, ..., Xn are independent and identically distributed
according to a Normal distribution, then the random variables:
X1 − µ
σ
, . . . ,
Xn − µ
σ
(4.7.860)
are also independent and identically distributed according to a Normal distribution but a cen-
tered reduced one.
Since:
nS2
∗
σ2
= χ2
(n) (4.7.861)
rearranging we get:
σ2
=
nS2
∗
χ2(n)
(4.7.862)
So on the population of measures, the true standard deviation follows the relationship given
above. It is therefore feasible to make statistical inference on the standard deviation When the
theoretical mean is known (...).

Draft
Since the chi-square distributions is not symmetric, the only way to make this inference is to
use numerical calculations and then we denote the conﬁdence interval at the level of 95% (for
example ...) as follows:
nS2
∗
χ2
2.5%(n)
σ2 nS2
∗
χ2
97.5%(n)
(4.7.863)
Either by writing 95% = 1 − α:
nS2
∗
χ2
α/2(n)
σ2 nS2
∗
χ2
1−α/2(n) (4.7.864)
the denominator being obviously the quantile of the chi-2 distribution. This relation is rarely
used in practice as the theoretical average (mean) is not known. In order to avoid confusion, the
latter relation is often denoted as follows:
nS2
∗
χ2
α/2,n
σ2 nS2
∗
χ2
1−α/2,n
(4.7.865)
Let’s see the most common case:
4.7.9.3 C.I. on the Variance with empirical Mean
Let us now make statistical inference when the theoretical average of population (i.e. the mean)
is not known. To do this, consider now the sum of:
χ2
(n) =
n
i=1
Xi − µ
σ
2
=
1
σ2
n
i=1
(Xi − µ)2
=
1
σ2
n
i=1
(Xi − ¯X) + ( ¯X − µ)
2
(4.7.866)
where for recall is the empirical average (arithmetic mean) of the sample:
¯X =
1
n
n
i=1
Xi (4.7.867)
Continuing the development we have:
χ2
(n) =
1
σ2
n
i=1
(Xi − ¯X)2
+ 2( ¯X − µ)
n
i=1
(Xi − ¯X) + n( ¯X − µ)2
(4.7.868)
However, we have proved earlier in this chapter that the sum of the deviations from the mean
was zero. So:
χ2
(n) =
1
σ2
n
i=1
(Xi − ¯X)2
+ 2( ¯X − µ) · 0 + n( ¯X − µ)2
=
1
σ2
n
i=1
(Xi − ¯X)2
+ n( ¯X − µ)2
=
n
i=1
(Xi − ¯X)2
σ2
+
n( ¯X − µ)2
σ2
=
n
i=1
(Xi − ¯X)2
σ2
+
¯X − µ
σ2/
√
n
2
(4.7.869)

Draft
and by taking back the unbiased estimator of the Normal distribution (we change notation to
respect the traditions and differentiate the empirical average of the theoretical mean):
S2
=
1
n − 1
n
i=1
(Xi − ¯X)2
(4.7.870)
Thus:
χ2
(n) =
n
i=1
(Xi − ¯X)2
σ2
+
¯X − µ
σ2/
√
n
2
=
(n − 1)S2
σ2
+
¯X − µ
σ2/
√
n
2
(4.7.871)
or with another common notation:
χ2
(n) =
(n − 1)S2
σ2
+
Mn − µ
σ2/
√
n
2
(4.7.872)
Since the second term (squared) follows a Normal centered reduced distribution too, so if we
remove it we get by the proof made above about the chi-square distribution following property:
χ2
(n − 1) =
(n − 1)S2
σ2
(4.7.873)
These developments allow us this time to also make inferences about the variance of a N(0, 1)
distribution when the parameters µ and σ of the parent population are both unknown. It is this
result that gives us, for example, the conﬁdence interval:
(n − 1)S2
χ2
α/2(n − 1)
σ2 (n − 1)S2
χ2
1−α/2(n − 1) (4.7.874)
when the theoretical average (mean) µ is unknown. And also to avoid any confusion, it is more
usual to write:
(n − 1)S2
χ2
α/2,n−1
σ2 (n − 1)S2
χ2
1−α/2,n−1
(4.7.875)
In the same way as above, we can calculate the standard deviation of the standard deviation that
has a great importance in the practice of ﬁnance:
V
(n − 1)S2
σ2
= V(χ2
n−1
(n − 1)2
σ4
V(S2
) = 2(n − 1)
V(S2
) =
2σ4
n − 1
σS2 =
2
n − 1
σ2
(4.7.876)
4.7.9.4 C.I. on the Mean with known unbiased Variance
We have proved much higher that the Student distribution came from the following relation:
T(k) =
Z
U/k
(4.7.877)

Draft
if Z and U are independent random variables and if Z follows a Normal centered reduced
distribution N(0, 1) and U a chi-square distribution χ2
(k) as:
fT (t) =
Γ
n + 1
2
Γ(n/2)
√
kπ
1 +
t2
n
−
k + 1
2
(4.7.878)
and remember that its density function is symmetrical!
Here is a very important application of the above result:
Suppose that X1, ..., Xn is a random sample of size n from a distribution N(µ, σ). So we can
already write that following developments made above:
Z =
¯X − µ
σ/
√
n
(4.7.879)
And for U that follows a χ2
(k) distribution, then if we ask that k = n − 1 then according to the
results above:
U =
(n − 1)S2
σ2
= χ2
(n − 1) (4.7.880)
We then get after some trivial simpliﬁcations:
Z
U/k
=
X − µ
σ/
√
n
(n − 1)S2
σ2
/(n − 1)
=
X − µ
1/
√
n
(n − 1)S2
(n − 1)
=
X − µ
1/
√
n
√
S2
=
¯X − µ
S/
√
n
(4.7.881)
So since:
T(k) =
Z
U/k
(4.7.882)
follows a Student distribution with parameter k then we get the independent one-sample t-test
or simply calld one-sample t-test:
¯X − µ
S/
√
n
= T(n − 1) (4.7.883)
which also follows a Student distribution of parameter n − 1 and is widely used in laboratories
for calibration testing.
This gives us also after rearrangement:
µ = ¯X −
S
√
n
T(n − 1) (4.7.884)
This allows us to make inference about the mean µ of a Normal distribution with the theoretical
standard deviation being unknown (meaning that there is not enough experimental values) but

Draft
where the unbiased estimator of the standard deviation is known. It is this result that gives us
the conﬁdence interval:
¯X −
S
√
n
Tα/2(n − 1) µ ¯X +
S
√
n
Tα/2(n − 1) (4.7.885)
where we see the same factors as for the statistical inference on the average (mean) of a (theo-
retical) random with know standard deviation as the Student distribution is asymptotically equal
to the Normal distribution for large values of n. Thus, the previous interval and the following
interval:
¯X − Zα/2
σ
√
n
µ ¯X + Zα/2
σ
√
n
(4.7.886)
givers very similar values (to three decimal places) for values of n at around 10, 000 (in practice
we consider that for 100 this is the same...).
We immediately deduce by the stability property of the chi-square distribution (proved above in
that this property arises from the Gamma distribution) the following test that we ﬁnd in many
statistical software:
( ¯X2 − ¯X1) − Tα/2(n − 2)
S2
1
n1
+
S2
2
n2
≤ µ2 − µ1 ≤ ( ¯X2 − ¯X1) + Tα/2(n − 2)
S2
1
n1
+
S2
2
n2
(4.7.887)
named bilateral t (Student) test on the difference of two means or more simply two sample
t-test.
We can of course therefore also determine the probability that the mean is inside or outside a
certain range ... (the both case being widely used in industry).
The reader can for fun control with Microsoft Excel 11.8346 that for a large number of mea-
surements n, the Student distribution tends to the Normal centered reduced distribution by com-
paring the values of the two functions below:
a„Fsx†@S7GPDnEIA
axy‚wFƒFsx†@S7GPA

Draft
Remark
The previous result was obtained by William S. Gosset around 1910. Gosset who had
studied mathematics and chemistry, worked as a statistician for the Guinness brewery
in England. At that time, we knew that if X1, ..., Xn are independent and identically
distributed random variables then:
¯X − µ
σ/
√
n
= N(0, 1) (4.7.888)
However, in statistical applications we were rather obviously interested in the following
quantity:
¯X − µ
S/
√
n
(4.7.889)
We then merely assume that this amount followed almost a Normal centered reduced
distribution, which was not a bad approximation as can show the image below (df =
n − 1):
Figure 4.103 – Comparison between the Normal and the Student distribution functions
After numerous simulations, Gosset came to the conclusion that this approximation was
valid only when n is large enough (so that gave him the indication that there must be
somewhere behind the central limit theorem). He decided to determine the origin of the
distribution and after completing a course in statistics with Karl Pearson he obtained his
famous result that he published under the pseudonym Student. Thus, is why we call
Student distribution that law that should have been called the Gosset distribution.
Finally, note that the Student’s T-test is also used to identify whether changes (increasing or

Draft
vice versa) in the average of two identical populations are statistically significant. That is to
say, if the size of two dependent samples is the same then we can create the following test (we
included all different types of writing that can be found in the literature and in many software
implementing this test):
T(n − 1) =
¯X − µ
S/
√
n
=
X ¯D−µD
SD/
√
n
=
¯D − δ0
SD/
√
n
=
1
n i
di − δ0
SD/
√
n
=
1
n i
(Xi2 − Xi1) − δ0
SD/
√
n
=
1
n i
Xi2 −
1
n i
Xi1 − δ0
SD/
√
n
=
( ¯X2 − ¯X1) − δ0
SD/
√
n
(4.7.890)
With:
SD =
1
n − 1 i
(di − ¯D) (4.7.891)
The prior-previous relation is very useful for comparing the same sample twice in different
measurement situations (sales before or after a discount on an article for example). This prior-
previous relation is called T-test (Student) averages two paired samples (or dependent sam-
ples) or more simple paired sample T-test.
Definition: We speak of paired samples if the sample values are taken 2 times on the same
individuals (i.e. the values of the pairs are not independent, unlike two samples taken indepen-
dently).
4.7.9.5 Binomial exact Test
Often when measuring we want to compare two small samples taken randomly (without replace-
ment!) from also a small population ... to know if they are statistically significantly different or
not as when we were expecting a perfect equality!
We are looking for a suitable test for the following cases:
• To know if the sample of a population prefers to use a given technical method of work
rather than another when we expect that the population does not prefer one of the other
• To know if the sample of a population has a predominant characteristic among two pos-
sibilities when we expect that the population is well balanced
Before going further into details, let us remind that we must be extremely cautious about how
to get the two samples. The experience must be unbiased, this is to say for reminder, that the
sampling protocol must not favor one of the both characteristics of the population (if you study

Draft
the balance between man/woman in a population by attracting people for the survey with a gift
in the form of jewelry or just by calling during the workdays you will have a biased sample ...
because you’ll probably naturally have more women than men...).
This said, this situation match with a binomial distribution for which we proved earlier in this
chapter that the probability of k successes in a population of size N with a probability of success
is p (probability of failure q being therefore 1 − p) was given by the relation:
P(N, k) = CN
k pk
qN−k
= CN
k pk
(1 − p)N−k
(4.7.892)
In the case before we are interesting we have p = q = 0.5:
P(N, k) = CN
k 0.5k
0.5k
= 0.5N
CN
k (4.7.893)
while remembering that the distribution will not be symmetrical and especially if the population
size N is small.
If we now denote by x the number of successes (considered as the size of the first sample) and
y is the number of failures (considered as the size of the second sample), then we have:
P(N, k) = 0.5N
CN
k = P(x + y, k) = 0.5x+y
Cx+y
k (4.7.894)
This being done, to build the test and by the asymmetry of the distribution, we will calculate
the cumulative probability that k is smaller than the x obtained by the experience and sum it
to the cumulative probability that k is greater than the y obtained by the experiment (which
corresponds to a cumulative probability of respectively left and right tails of the distribution).
So this sum corresponds to the probability:
P = 0.5N
x
k=0
CN
k + 0.5N
N
k=y
CN
k (4.7.895)
and this last relation is called binomial exact test (two-tailed).
If the probability P obtained by the sum is above a certain cumulative probability fixed in ad-
vance, then we say that the difference with a random sample in a perfectly balanced population
is not statistically significant (bilaterally ...) and respectively if it is below, the difference will
be statistically significant and therefore we reject the assumed equilibrium.
Therefore, if:
0.5N
x
k=0
CN
k + 0.5N
N
k=y
CN
k = 0.5N
x
k=0
CN
k +
N
k=y
CN
k ≥ α (4.7.896)
the difference with a balanced population will be considered not statistically significant. Of-
ten we will α to be at the maximum equal to 5% (but rarely below) which corresponds to a
confidence interval of 95%.
Unfortunately from a statistical software to the other the required parameters or results will not
necessarily be the same (spreadsheets softwares for example do not include a specific function
for the binomial test, will often have to build a table or develop yourself a function). For

Draft
example, some software automatically calculate and impose (which is quite logical in a sense
...)
0.5N
x−1
k=0
CN
k +
N
k=y+1
CN
k = P (4.7.897)

Draft
Example:
From a small population having two particular characteristics x and y that interest us
and which we expect to have a perfect balance but as x = y we actually got x = 5 and
y = 7. We would like do the calculation with Microsoft Excel 11.8346 to know whether
this difference is statistically significant or not at a level of 5%?
So, to answer this question, we will calculate the cumulative probability:
0.5N
k=0
CN
k +
N
k=y
CN
k = 0.512
5
k=0
C12
k +
12
k=7
C12
k (4.7.898)
which gives us:
Figure 4.104 – Calculated values of the binomial coefficients in Microsoft Excel 11.8346
thus explicitly:
Figure 4.105 – Formulas for calculating binomial coefficients in Microsoft Excel 11.8346
thus the cumulative probability being 0.774 (i.e. 77.4%) the difference compared with
balanced population will be considered as not statistically significant.

Draft
Remark
This test is also used by most statistical software (such as Minitab) to give a confidence
interval of the conformity of opinions in relation to that of an expert. This is what we call
an R R study (reproductability repeatability) by attributes (see my book on Minitab
for an example).
4.7.9.6 C.I. for a Proportion
For information some statisticians use the fact that the Normal distribution arises from the Pois-
son distribution which itself derives from the binomial distribution (we have proved it when n
tends to infinity and p and q are of the same order) to build a confidence interval in the context
of the analysis of proportions (widely used in the analysis of the quality in the industry).
To see this, we note Xi the random variable defined by:
Xi =
1 if the i element of the sample has the attribute A
0 otherwise
(4.7.899)
(4.7.900)
where the attribute A can be the property defective or non-defective for example, in an
analysis of pieces. We note by K the number of successes of the attribute A.
The random variable X = X1 + X2 + ... + Xn we have proved it earlier in this chapter, follows
a binomial distribution with parameters n and p with the moments:
µ = E(X) = np
σ = V(X) =
√
npq = np(1 − p)
(4.7.901)
That said, we do not know the true value of p. We will use the estimator of the binomial
distribution proved above:
ˆp =
k
n
=
X
n
(4.7.902)
Based on the properties of the mean we have then:
E(ˆp) = µˆp = E
X
n
=
np
n
= p (4.7.903)
And by using the properties of the variance, we have following relation for the variance of the
sample mean of the proportion:
V(ˆp) = σ2
ˆp = V
X
n
=
V(X)
n2
=
np(1 − p)
n2
=
p(1 − p)
n
(4.7.904)

Draft
This then brings us to:
µˆp
∼= ˆp and σˆp
∼=
ˆp(1 − ˆp)
n
(4.7.905)
Finally, remember that we have proved that the normal distribution resulted from the binomial
distribution under certain conditions (practitioners admit that it is applicable as n 50 and
np ≥ 5). In other words, the random variable X following a binomial distribution follows a
Normal distribution under certain conditions. Obviously, if X follows a Normal distribution
then X/n also (and so do ˆp...). Therefore we can center and reduce ˆp so that it behaves as the
reduced Normal centered random variable denoted by Z:
Z =
ˆp − p
¯p(1 − ¯p)
n
(4.7.906)
Examples:
E1. If 5% of the annual production of a business fails, what is the probability
that by taking a sample of 75 pieces of the production line only 2% or less will be
defective?
We therefore have:
Z =
0.02 − 0.05
0.05 · 0.95
75
(4.7.907)
The corresponding cumulative probability to that value can be easily obtained with Mi-
crosoft Excel 11.8346:
axy‚wƒhsƒ„@EIFIWAaIIFTT7
But note that we do not have np ≥ 5 that is satisﬁed therefore we could exclude to use
this result.
E2. In its report from 1998, JP Morgan explained that during the year 1998 its losses
went beyond the Value at Risk (see section Economy) 20 days on 252 working days of
the year based on a 95% temporal VaR (thus 5% of working days considered as loss). At
the threshold of 95% it is just bad luck or is that the VaR model used was bad?
Z =
ˆp − p
¯p(1 − ¯p)
n
=
nˆp − np
ˆp(1 − ˆp)n
=
−0.05 · 252 − 20
0.05(1 − 0.05)252
∼= −2.14 −1.96
(4.7.908)
So it was just bad luck.
We can now approximate the conﬁdence interval for the proportion by using the fact that the
binomial distribution has a Normal asymptotic behavior under the conditions demonstrated dur-
ing our introduction of the Normal distribution such as we get the one-proportion Z test or

Draft
also called one-proportion p test:
ˆp − Zα/2
ˆp(1 − ˆp
n
≤ p ≤ ˆp + Zα/2
ˆp(1 − ˆp
n
(4.7.909)
Before proceeding to an example, it may be useful to clarify to the reader that this approximation
by a Normal distribution is very common and that we’ll meet it again numerous times in proofs
that will follow. It is even so common that this approximation method has a name..: the Wald
method (well actually there are several Wald methods but we will only use the most known
one).
Example:
We take α = 5%, then we have:
ˆp − Z0.025
ˆp(1 − ˆp
n
≤ p ≤ ˆp + Z0.025
ˆp(1 − ˆp
n
(4.7.910)
That is to say:
ˆp − 1.96
ˆp(1 − ˆp
n
≤ p ≤ ˆp + 1.96
ˆp(1 − ˆp
n
(4.7.911)
On production of 300 elements we found that 8 were defect. What is the confidence
interval?
We check first with:
ˆp =
8
300
∼= 2.66% (4.7.912)
that:
nˆp = 300
8
300
= 8 ≥ 5 (4.7.913)
So it is acceptable to use the confidence interval by the Normal distribution. We therefore
have:
8
300
− 1.96
8
300
1 −
8
300
300
≤ p ≤
8
300
+ 1.96
8
300
1 −
8
300
300
(4.7.914)
Thats is to say:
0.84% ≤ p ≤ 4.49% (4.7.915)
To conclude this subject, we can obviously be interested to the number of individuals (sam-
ple size) necessary to satisfy a certain (imposed) confidence interval accuracy when having an
imposed standard deviation.

Draft
We therefore have according to the above assumptions and in the acceptance of the approxima-
tion by a Normal distribution:
Z =
ˆp − p
pq
n
=
δ
p(1 − p)
n
(4.7.916)
And by proceeding in an identical manner to developments made above with the Normal distri-
bution, we obtain:
n ∼=
Z2
α/2p(1 − p)
δ2
=
Z2
α/2pq
δ2
(4.7.917)
obviously we normally take the integer value in practice...
A question that often comes up in practice is the fact to know whether you have to take a
unilateral or bilateral test. In fact there is no precise answer, it depends on what we want to
highlight.
Remark
The size of the parent population for the relations developed above does not come into
consideration in the calculation of confidence intervals or in one of the sample size, and
because it is considered infinite. So be careful to not have sometimes sample sizes that
are larger than the possible real parent population...
Example:
We would like to know the number of individuals (sample size) to take in a pro-
duction lot knowing that the proportion of defective units is imposed at 30% with a
tolerated error of about 5% between the actual and empirical proportion and to obtain a
confidence interval at a level of 95% of the result:
n ∼=
Z2
5%/230%(1 − 30%)
5%2
=
(−1.96)2
30%70%
5%2
∼= 322 (4.7.918)
Remark
The last relation is very often used in sampling theory (analysis for referendum with
responses of type: Yes/No) where sometimes the sample size n is imposed for costs
reasons of the survey and for which we seek to calculate the uncertainty δ and sometimes
the reciprocal (the uncertainty is imposed and therefore we seek to know the sample size).

Draft
4.7.9.6.1 Test of equality of two Proportions
Always in the same context as the previous approximation of the binomial distribution
by a Normal distribution, the industry (especially biostatistics) likes to compare two pro-
portions of two different populations to see if they are statistically equal or not (i.e. said
statistically significantly different or not).
Therefore, let us recall that we have proved the stability of the Normal distribution if two random
variables are independent and identically distributed (according to a Normal distribution!):
X − Y = N µ = µX − µY , σ = σ2
X + σ2
Y (4.7.919)
Under the above assumptions it is then approximately the same for the difference of two pro-
portions:
X
n1
−
Y
n2
∼= N µ¯pX
− µ¯pY
, σ2
¯pX
+ σ2
¯pY
∼= N ˆpX − ˆpY , σ2
ˆpX
+ σ2
ˆpY
= N

ˆpX − ˆpY ,
ˆpX(1 − ˆpX)
n1
+
ˆpY (1 − ˆpY )
n2


(4.7.920)
Therefore we know that this new reduced centered variable follows a Normal distribution as:
Z =
(ˆpX − ˆpY ) − µ
ˆpX(1 − ˆpX)
n1
+
ˆpY (1 − ˆpY )
n2
(4.7.921)
and as we seek to know the cumulative probability that the mean of the difference is zero, the
latter relation is reduced in this case to:
Z =
ˆpX − ˆpY
ˆpX(1 − ˆpX)
n1
+
ˆpY (1 − ˆpY )
n2
(4.7.922)
Obviously we can build (as always...) a confidence interval from this relationship.
However, it seems that the latter approximate relation following the return on experience is
more correct when taking a modified denominator:
Z =
ˆpX − ˆpY
ˆpX(1 − ˆpX)
n1
+
ˆpY (1 − ˆpY )
n2
=
ˆpX − ˆpY
ˆp(1 − ˆp)
1
n1
+
1
n2
(4.7.923)
which ˆp will be taken as the mixture of the two populations. That is to say:
ˆp =
sum of events in each of the two samples
sum of samples sizes
(4.7.924)
thus (by changing the notations of the indices of the experimental proportions):
ˆp =
n1 ˆp1 + n2 ˆp2
n1 + n2
(4.7.925)

Draft
This test is named two proportions Z test or more simply two proportions p test. In
medicine, it is named the test of differences in risk (meaning implicitly that each proportion
is a segment of the studied population in relation to an undesired event).
Example:
In the context of a sampling plan (see section Industrial Engineering) we have
taken on a first batch of 50 individuals, 48 that are in perfect conditions. In a second
batch of 30 individuals, 26 were in perfect condition.
Thus we have:
ˆpX =
48
50
= 96%, ˆpY =
26
30
= 86.66% (4.7.926)
We would like to know if the difference is statistically significant with a 95% confidence
interval or simply due to chance. We then use:
ˆp =
n1 ˆp1 + n2 ˆp2
n1 + n2
=
48 + 26
50 + 30
= 92.50% (4.7.927)
and:
Z =
ˆpX − ˆpY
ˆp(1 − ˆp)
1
n1
+
1
n2
=
0.96 − 0.8666
0.925(1 − 0.925)
1
50
+
1
30
∼= 1.535 (4.7.928)
This corresponds to a cumulative probability using Microsoft Excel 11.8346:
axy‚wƒhsƒ„@IFSQSAaWQFUU7
Therefore the difference is due to chance (that said it is almost in extremis...). In other
words, it is not statistically significant under the set constraints.

Draft
4.7.9.7 Sign Test
We measure something on a sample and later, we measure the same thing on this same sample
but with a different method (so it is therefore paired samples!). Both ordered rankings of mea-
sures are compared and too each case is assigned a sign (+ for an increase in the rankings, -
in case of descent). Those who remain at the same level are eliminated.
According to the hypothesis to be tested, there are so many + as -, that is to say, the median
of the distribution has not changed (this statement may not seem obvious at ﬁrst reading so be
aware to take time to think about it).
The idea is that for each pair of values, there are only two possible signs of change, we have a
chance on two (50% probability) that the difference is positive or negative. This test is based
only on the study of signs of the differences between the pairs of individuals, regardless of the
values o these differences.
We can wish to control two assumptions:
• The inequality of proportions of signs must be statistically signiﬁcant. So one of the two
signs must be in a small number compared to the other, which corresponds to a left-sided
test (the cumulative probability of the small number of characters must be below a certain
level α).
• The proportion of the two signs must be low unbalanced (P(+) = P(−) = 0.5). It is
therefore in this case a bilateral test (the most common case) with a given level α.
To create such a test, we consider the appearance of the + and - as a binary random sam-
pling system where the order of success is not taken into account (it is therefore based on a
binomial or hypergeometric distribution) and with replacement (which immediately eliminates
the hypergeometric distribution that is not symmetric and problematic to use in practice...). To
consider a random sampling with replacement (with the fact that we do not reinject each in-
dividual in reality), it is necessary that the population N is large. This is why the sign test
considers that the paired values should be continuous (which allows verbatim to approach the
hypergeometric distribution by the binomial distribution). However, some statistical software
use the hypergeometric distribution for the sake of precision.
Remark
You should know that most statistical software, do implicitly the assumption in this test
that the data are continuous and use therefore the binomial distribution.

Draft
Example:
Consider two sets of measurements with two different methods. We would like to
test the hypothesis with a confidence level α of 5% if the difference between the two
methods is statistically significant (thus we expect a balance of signs). This is therefore
two samples sign test (knowing that it is also possible to do the same by comparing the
values of a single sample to its median):
20.4, 25.4, 25.6, 25.6, 26.6, 28.6, 28.7, 29, 29.8, 30.5, 30.9, 31.1
20.7, 26.3, 26.8, 28.1, 26.2, 27.3, 29.5, 32, 30.9, 32.3, 32.3, 31.7
(4.7.929)
Therefore we have the differences:
−0.3, −0.9, −1.2, −2.5, 0.4, 1.3, −0.8, −3.0, −1.1, −1.8, −1.4, −0.6 (4.7.930)
With the signs:
−, −, −, −, +, +, −, −, −, −, −, − (4.7.931)
Well it is already clear that the result will be the rejection of the hypothesis as there is no
difference. But we will still do the calculations. As the test is a two-sided at the level of
5% , the cumulative probability of obtaining at least two signs + must not be less than
2.5% and not more than 97.5% if we want to accept (not reject) the assumption as that
the difference is not statistically significant.

Draft
We then have:
P(X ≤ 2) =
2
k=0
C12
k 0.5k
(1 − 0.5)12−k
=
2
k=0
C12
k 0.5k
0.512−k
= 0.512
2
k=0
C12
k
∼= 1.93%
(4.7.932)
Either with Microsoft Excel 14.0.6123:
afsxywhsƒ„@PDIPDHFSDIAaIFWPV7
or if we don’t do the approximation by being more accurate with the hypergeometric
distribution:
ar‰€qiywFhsƒ„@PDPRGPDIPDPRD„‚…iAaHFIU7
which is not really better...!
So the cumulative probability is less than 2.5% and is by far not more than 97.5%, there-
fore we reject the hypothesis as that the difference is not statistically significant.
We could accept the hypothesis if we take for α the value:
p − value ∼= 1.93% · 2 = 3.86% (4.7.933)
but this is not the case!
To conclude on this sign test (median test), we have for information some statistical
software propose a confidence interval of the median based on the calculation method
described previously (confidence interval of the binomial distribution). However, we
believe that it would be to use bootstrapping as we have seen in the chapter on Numerical
Methods, so won’t introduce this technique here. In addition it may be useful to know
that some make an approximation using the Normal distribution (as with most tests but
we won’t study this approximation in this context).
4.7.9.8 Mood’s Median Test
Here we will introduce a test that has many names: median test, Mood’s median test or
Westenberg-Mood’s mediant test or Brown-Mood’s median test ...
We consider two independent samples (X1, ..., Xn1 ) and (Y1, ..., Yn2 ). We assume that
(X1, ..., Xn1) is an independent and identically distributed sample from a continuous distribu-
tion F and (Y1, ..., Yn2 ) is an independent and identically distributed sample from a continuous
distribution G.
After the grouping of the n1 + n2 values of the two samples, k = n1Mn is the number of obser-
vations Xi of the first sample that are greater than the median N = n1 + n2 of the observations
(the notation is not great because it can give the impression that is is a multiplication...).

Draft
Under the null hypothesis that X and Y variables follow the same continuous distribution (that
is to say, G = F) hypothesis, the variable k = n1Mn can take the values 0, 1, ..., n1 according
to the hypergeometric distribution:
P(N, N/2, n1, k) =
Cn1
k Cn2
N/2−k
CN
N/2
=
n1!
k!(n1 − k)!
n2!
(N/2 − k)!(n2 − (N/2 − k))!
N!
N/2!(N − N/2)!
(4.7.934)
Therefore, we can calculate the unilateral cumulative probability of having k. Mood’s test is
also a purely unilateral test.
Example:
Consider the two samples:
23.4, 24.4, 24.6, 24.9, 25.0, 26.2, 26.3, 26.8, 26.8, 26.9, 27.0, 27.6, 27.7
22.5, 22.9, 23.7, 24, 24.4, 24.5, 25.3, 26, 26.2, 26.4, 26.7, 26.9, 27.4
(4.7.935)
The overall median calculated with Microsoft Excel 14.0.6123 is 26.10. We have a total
of:
k = n1Mn = 8 (4.7.936)
Then it comes with Microsoft Excel 14.0.6123:
ar‰€qiywFhsƒ„@VDPTGPDIQDPTD„‚…iAaWRFPR7
So at a threshold of 5%, we do not reject the null hypothesis (but... being close to the
limit this is a bit tedious to conclude that ...). If we do the same calculation using the
binomial distribution we obtain:
afsxywFhsƒ„@VYPTGPYHFSYIAaVTFTS7
But obviously here the approximation does not apply since a binomial approximation is
acceptable in practice when the sample is about 10 times smaller than the population.
Remark
Unfortunately, there are several versions of the Mood test. For example, a software such
as Minitab compares thanks to a contingency table... the values above or below the
median and made a simple chi-square test of independence (Pearson test) as seen in the
Chapter of Numerical Methods.

Draft
4.7.9.9 Poisson Test (1 sample)
We know that a number of rare events follow a Poisson distribution. We can then allow us as
for any other distribution to calculate the cumulative probability in a given interval (bilateral or
unilateral).
So if we have a discrete random variable following a Poisson distribution:
Pk =
µk
k!
eµ
(4.7.937)
We then have to a certain right sided level of confidence α, the closest value n of k satisfying
the condition:
n
k=0
µk
k!
eµ
→ 1 − α (4.7.938)
So to find the value n (strictly positive integer or null value) we should reverse the sum, which
is not... something funny to do (this is why most spreadsheet software do not offer at this day
the inverse of the Poisson distribution).
Now recall that we have seen in the section on Sequences And Series, the following Taylor
(Maclaurin) series with full integral rest to order n − 1 around 0 to λ:
n−1
k=0
λk
k!
eλ
= 1 −
λˆ
0
1
2k/2−1Γ(k/2)
xk/2−1
e−x/2
dx (4.7.939)
Result we had also given in the form of functions for Microsoft Excel 14.0.6123 so that the
reader can verify this equivalence:
a€ysƒƒyx@x ∈ N, µ,„‚…iA
aIEgrshsƒ„@2µ, 2(x + 1),„‚…iA
Then it follows that in spreadsheets softwares, we can use the inverse chi-square distribution
to calculate the inverse of the Poisson distribution with this time however a small nuance: the
result will not necessarily be an integer.
If for example we take (always with Microsoft Excel 14.0.6123)
aIEgrsFhsƒ„@PBPHDPB@ISCIAD„‚…iAaISFTSIQIQS7
The question is then to find the notation for the opposite ... This is then given by (we divide by
2 to fall back on the mean that is the value of interest):
agrssx†@IEISFTSIQIQS7DPB@ISCIAAGPaISFSQIWRPSV

Draft
Finally, the notation of the inverse is relatively natural. Thus, the 1 sample Poissons’ test at a
given right sided α level can be written:
k ≤agrssx†@IE—lph—DPB@num˜er of me—suresCIAAGP
Formally:
λ ≤
χ2
1−α2(nchoosed + 1)
2
(4.7.940)
Note however one thing! It seems that some statistical software approximate sometimes with
abuse the Poisson distribution by a Normal distribution. Therefore, the unilateral interval is
calculated with:
µ ≤ ¯X + Z1α
σ
√
n
(4.7.941)
But with the Poisson distribution, remember that we have:
µ = σ2
= λ (4.7.942)
¯X = ˆλ (4.7.943)
Therefore:
λ = ˆλ + Z1−α
λchoosed
n
(4.7.944)

Draft
Examples:
A company manufactures televisions in a constant quantity and has measured the
number of defective product produced each quarter for the past ten years (so 4 times
10 measures). The stakeholders determines the maximum acceptable number of
defective units is 20 per quarter and wants to determine if the production satisﬁes these
requirements (under the assumption that the distribution of defective follow a Poisson
distribution) at a conﬁdence level of 5%.
The 40 measures give us an average of:
¯X = ˆλ = 17.825 (4.7.945)
Then we have with the rough approximation:
λ = 17.825 + Z1−5%
20
40
(4.7.946)
Either in a spreadsheet software like Microsoft Excel 14.0.6123:
axy‚wFƒFsx†@IES7ABƒ‚„@PHGRHACIUFVPSaIVFWVV
or:
λ ≤
χ2
1−5%2(20 + 1)
2
(4.7.947)
Either in a spreadsheet software like Microsoft Excel 11.8346:
agrssx†@IES7DPB@PHCIAAGPaIRFHUP
Obviously, in the bilateral case, we have:
χ2
α/22(n + 1)
2
≤ λ ≤
χ2
1−α/22(n + 1)
2
(4.7.948)

Draft
Example:
An airline company had 2 crash on 1, 000, 000 flights (very rare event). What is
two-sided the confidence at the level of 95% knowing that globally the number of
accidents per million is 0.4.
We have therefore:
χ2
5%/22(2 + 1)
2
≤ λ ≤
χ2
1−5%/22(2 + 1)
2
(4.7.949)
Either the upper bound with a spreadsheet software like Microsoft Excel 11.8346 is given
by:
agrssx†@IES7GPDPB@PCIAAGPaUFPPR
and for the lower bound:
agrssx†@IES7GPDPB@PCIAAGPaHFTIV
So statistically, the company is less secure than all companies.
4.7.9.10 Poisson Test (2 samples)
We have just seen that:
χ2
α/22(n + 1)
2
≤ λ ≤
χ2
1−α/22(n + 1)
2
(4.7.950)
However, following the same reasoning that led us to construct the following test of average
comparison:
( ¯X2 − ¯X1) − Zα/2
σ2
1
n1
+
σ2
2
n2
≤ µ2 − µ1 ≤ ( ¯X2 − ¯X1) + Zα/2
σ2
1
n1
+
σ2
2
n2
(4.7.951)
or its equivalent with the Student distribution when the true standard deviation is not known
and using the fact that we have shown that the Poisson distribution is stable by the addition (and
hence by subtraction), that Gamma distribution was also stable by the addition (and therefore
by subtraction) and also the chi-square distribution since it is only a special case of the gamma
distribution, we tend to write perhaps a little bit to fast the extension of what we have seen
before:
χ2
α/2(2(n2 + 1) − 2(n1 + 1))
2
≤ λ2 − λ1 ≤
χ2
1−α/2(2(n2 + 1) − 2(n1 + 1))
2
(4.7.952)
And in the facts this is a trap as say some practitioners ... because the chi-square distribution has
a support which is defined as being strictly positive and the confidence interval could naturally

Draft
have a negative left terminal (... O_o). One solution could be to use the test of the difference of
two proportions that we have already discussed earlier:
Z ∼=
(ˆpX − ˆpY ) − µ
ˆpX(1 − ˆpX)
n1
+
ˆpY (1 − ˆpY )
n2
(4.7.953)
Of course, only in the case that the conditions for approaching the test with a Normal distribu-
tion are met (proportions have to be typically less than 0.1 and no greater than 50).
Most softwares seem to have implemented this latter method (with which I do not necessarily
agree).
Example:
An airline company had 2 airplanes crash was in 1, 000, 000 flights (very rare
event). Another company had 3 crashes in 1, 200, 000 flights. What is the two sided
confidence interval at the level of 95% assuming that the difference is zero.
Therefore, the proportions are:
ˆpX =
2
n1
=
2
1, 000, 000
= 0.000002 and ˆpY =
3
n2
=
3
1, 200, 000
= 0.0000025
(4.7.954)
We write:
∆ = ˆpX − ˆpY = −0.0000005
ˆqX = 1 − ˆpX
ˆqY = 1 − ˆpY
(4.7.955)
Then we have:
∆ − Z5%/2
ˆpX ˆqX
n1
+
ˆpY ˆqY
n2
≤ µ ≤ ∆ + Z5%/2
ˆpX ˆqX
n1
+
ˆpY ˆqY
n2
(4.7.956)
which gives a confidence interval for the expected theoretical proportion difference.
−0.000004461 ≤ µ ≤ 0.000003461 (4.7.957)
and therefore as −0.0000005 is in this interval, we accept the hypothesis as the difference
of proportions are not statistically significant at the threshold of 5%.
Or taking the non approximated expression, we have (with the same conclusion):
χ2
5%/2(2(3 + 1) − 2(2 + 1))
2
≤ λ2 − λ1 ≤
χ2
1−5%/2(2(3 + 1) − 2(2 + 1))
2
⇓
0.0253 ≤ µ ≤ 3.6889
So to summarize some convergence of distributions in all these different tests and intervals that

Draft
we have seen so far, we offer the reader the following diagram that we hope... will clarify
perhaps more or less things:
Figure 4.106 – Convergence of different customary distributions in elementary statistical inference
And also this table where all relationships have been demonstrated in detail above, and some
already used (others will be used later):
Statistical sampling Statistic Average Statistic Standard-Deviation
Mean
(infinite population)
µ
σ
√
n
Mean
(finite population)
µ
σ
√
n
N − n
N − 1
Proportion
(finite population)
p
p(1 − p)
n
Proportion
(infinite population)
p
p(1 − p)
n
N − n
N − 1
ˆσ2
(infinite population2
)
n − 1
n
σ2 2(n − 1)
n2
σ2
Table 4.28 – Table of statistical sampling proved earlier and used in part until now

Draft
4.7.9.11 Confidence/Tolerance/Prediction Interval
Here we go and in order to avoid frequent confusion and before moving on to more complex
subjects, we will compare the confidence interval, the tolerance interval (often called fluctua-
tion interval in some school programs) and finally the prediction interval.
Definitions:
D1. The tolerance interval (or fluctuation interval) is an interval containing a certain per-
centage (usually 68.26, 95.44or99.73% in the case of a Normal distribution) of individuals
in a population of measures.
D2. The confidence interval for a sample mean (or proportion p) contains the interval value
to a given confidence level (usually 90, 95 or 99% in the case two sided case) for the
expected mean (true average) or the proportion of the population.
D3. Or the prediction interval to determine an interval for a single value based on the knowl-
edge of the sample mean and the standard deviation of the population.
An example being more often better than a thousand words, consider the case where the mean
and the standard deviation of prices are 49 DVD are given by:
¯X = 31.5531 S = ˆσ = 0.7992 (4.7.958)
Therefore we have:
[ ¯X ± S] = [30.8, 32.4]
[ ¯X ± 2S] = [30.0, 33.2]
[ ¯X ± 3S] = [29.2, 34.2]
(4.7.959)
corresponding to the tolerance intervals according to a Normal distribution of 68.26, 95.44 and
99.73%.
But a a confidence interval of 95% based on the relationship proved above:
¯X −
S
√
n
Tα/2(n − 1) ≤ µ ≤ ¯X + −
S
√
n
Tα/2(n − 1) (4.7.960)
gives:
31.32 ≤ µ ≤ 31.78 (4.7.961)
So 95% cumulative probability that the true mean is between 31.32 and 31.78.

Draft
Figure 4.107 – Histogram of the prices of a sample of 49 DVD
Now lets us introduce a new concept that is rarely addressed in the statistics literature. The idea
of the prediction interval is rather than look at the conﬁdence interval of the mean based on an
experimental average, to use this experimental average (sample mean) as a basis for predicting
the interval of a single value (and no not of the average!).
We’ll look at the difference between the mean and a punctual value:
¯x − x (4.7.962)
that we will assume close to zero (it is better to have a reliable product and pass the tests to
obtain the authorization of sales...). About the variance, what interests us is not just the standard
deviation of the mean anymore, but the standard deviation of the difference... and as the sample
is independent of the unique value we have:
σ2
¯x−x = σ2
¯x + σ2
x =
σ2
n
+ σ2
= σ2
1 +
1
n
(4.7.963)
So we can write as a ﬁrst approximation:
Z ∼=
¯x − x
σ2
¯x−x
=
¯x − x
σ2 1 +
1
n
=
¯x − x
σ 1 +
1
n
(4.7.964)
And of course after what we saw:
T(n − 1) ∼=
¯x − x
S 1 +
1
n
(4.7.965)

Draft
So we can build verbatim the prediction interval:
¯x − Tα/2(n − 1)S 1 +
1
n
≤ x ≤ ¯x + Tα/2(n − 1)S 1 +
1
n
(4.7.966)
4.7.10 Weak Law of Large Numbers
We will now focus on a very interesting relation in statistics that can tell a lot of things while
having little information and whatever the distribution (which is not bad!). This is a widely used
property for example in statistical simulation in the context of the use of Monte-Carlo technics.
Given a random variable with values in R+
. Then we will show the following relation called
Markov inequality:
∀λ 0 : P(X λ)
E(X)
λ
(4.7.967)
with in the particular context of probabilities E(X) ≤ λ.
In other words, we propose to prove that the probability that a random variable is greater than
or equal to λ is less then or equal to its mean divided by the value λ considerated and regardless
of the probability distribution of the random variable X!
Proof 4.27.6. Let us write the values of X by (x1, ..., xn), where 0 ≤ x1 x2 ... xn (that
is to say sorted in ascending order) and let us also write x0 = 0. We note first that the inequality
is trivial in case where λ ≥ xn ≥ 0. Indeed, as X can be included only between 0 and xn by
definition then the probability that it is greater to xn is equal to zero. In other words:
P(X λ) = 0 (4.7.968)
and X being positive, the E(X) is also positive, thus the inequality in this special case in a first
time.
Otherwise, we have 0 λ ≤ xn and then there exist one k ∈ (1, ..., n) such that xk−1 λ ≤ xk.
Thus:
E(X) =
n
i=1
xipi
n
i=k
xipi
n
i=k
pi = λP(X λ) (4.7.969)
Q.E.D.

Draft
Example:
We assume that the number of outgoing parts of a given factory during one week
is a random variable of mean 50. If we wish to estimate the cumulative probability that
production exceeds 75 parts we will simply apply:
P(X 75)
E(X)
λ
=
50
75
= 0.6666 = 66.66% (4.7.970)
Consider now a kind of generalization of this inequality called Bienayme-Chebyshev inequal-
ity (abbreviated BC inequality) that will allow us to get a very very interesting and important
result a little bit later.
Consider a real random variable X (so we do not limit ourselves to the only cases where it is in
R+
). Then we will prove the following Bienayme-Chebyshev inequality:
∀ε 0 : P(|X − E(X)| ε)
V(X)
ε2
=
σ2
ε2
(4.7.971)
which expresses the fact that the smaller the standard deviation is, more the probability that the
random variable X moves away from its expectation is low.
Proof 4.27.7. We obtain this inequality by first writing:
∀ε 0 : P(|X − E(X)| ε) = P [X − E(X)]2
ε2
(4.7.972)
where the choice of the square will serve us for a future simplification.
Then by applying Markov’s inequality (as you see is something that can be useful ...) to the
random variable Y = [X − E(X)]2
with λ = 2
it comes automatically:
P(Y ε2
) = P [X − E(X)]2
ε2 E(Y )
ε2
=
E ([X − E(X)]2
)
ε2
(4.7.973)
Then, using the definition of the variance:
E [X − E(X)]2
= E [X − µX]2
= V(X) = σ2
(4.7.974)
We get:
P(Y ε2
)
E ([X − E(X)]2
)
ε2
=
V (x)
ε2
=
σ2
ε2
(4.7.975)
Q.E.D.
If we put:
= tσ (4.7.976)

Draft
The equality will be written:
P(|X − E(X)| tσ)
1
t2 (4.7.977)
and expresses the cumulative probability in order that X moves away from the mean of more
than t times its standard deviation, is below 1/t2
. There is, in particular, less than 1 chance on 9
that X moves away from its mean by more than three times the standard deviation. This is also
this theorem that is used by the Basel Committee to define the Value At Risk correction factor
used in finance (see section Economy).
Example:
We take the example where the number of outgoing parts of a given factory dur-
ing one week is a random variable of mean 50. We assume in addition that the variance
of the weekly production is 25. We seek to calculate the probability that the production
of next week is between 40 and 60 pieces.
|40 − 50| = |60 − 50| = 10 (4.7.978)
To calculate this we must first remember that the BC inequality is based in part on the
term |X − E(X)| thus we have:
−10 ≤ X − E(X) ≤ 10 (4.7.979)
Then we just have to apply numerical values to the inequality:
P(|X − E(X)| ε) = P(|X − E(X)| 10) =
V (x)
ε2
=
25
102
= 0.25 = 25%
(4.7.980)
The last two inequalities obtained before the example will allow us to obtain a very important
and powerful relation that we call weak law of large numbers (WLLN) or Khinchin theorem.
Consider a random variable X having a variance and (Xn)n∈N∗ a sequence of independent
random variables (i.e. uncorrelated by pairs ) of the same distribution as this of X and all
having the same mean µ and the same standard deviations σ.
What we will show is that if we measure the same random quantity Xn of the same distribution
in the process of a series of independent experiments (so in this case, we say technically that
the sequence (Xn)n∈N∗ of random variables is defined on the same probability space) , then the
arithmetic average of the observed values will stabilize on the mean of X when the number of
measurements tends to infinity.
∀ε 0,
1
n
n
i=1
Xi − µ ε
σ2
nε2
→
n→+∞
0 (4.7.981)
when n → +∞ this is the very important result which we did mention above! The empirical
estimator of the mean tends for any distribution to the true hope if n is large! So by the this

Draft
rule we assure that the sample average is a consistent estimator of mean! This result (quite
intuitive) is sometimes called the fundamental theorem of Monte-Carlo because it is central
to the simulations principle of the same name (see Numerical Methods), which are crucial in
the study of advanced statistics.
So in other words, the cumulative probability that the difference between the arithmetic average
and the expected of the observed random variables to be in a given range around the average
tends to zero as the number of measured random variables tends to infinity (that which is ulti-
mately intuitive).
This result allows us to estimate the expected mean value using the empirical mean (arithmetic
average) calculated on a very large number of experiments.
Proof 4.27.8. We use the Chebyshev-Bienaymé inequality for the random variable (this relation
is difficult to interpret but allows us to get the desired result):
Y =
1
n
n
i=1
Xi (4.7.982)
And we first calculate using the mathematical properties of the mean that we proved earlier:
E(Y ) = E
1
n
n
i=1
Xi =
1
n
E
n
i=1
Xi =
1
n
n
i=1
E(Xi) =
1
n
n
i=1
µ = µ (4.7.983)
and in a second time using the mathematical properties of the variance as already proved above:
V(Y ) = V
1
n
n
i=1
Xi =
1
n2
V
n
i=1
Xi =
1
n2
n
i=1
V(Xi) + 2
n
1 ij n
cov(Xi, Xj)
(4.7.984)
and since we assumed uncorrelated variables then the covariance between them is zero there-
fore:
V(Y ) =
1
n2
n
i=1
V((Xi) =
1
n2
n
i=1
σ2
=
1
n2
nσ2
=
σ2
n2
(4.7.985)
So by injecting it into the BT inequality:
P [Y − E(Y )]2
ε2 V (Y )
ε2
(4.7.986)
that becomes:
P

 1
n
n
i=1
Xi − µ
2
ε2

 σ2
nε2
(4.7.987)
and the inequality tends effectively to zero as well n at the denominator tends to infinity.
Q.E.D.

Draft
Note that the latter relation is often denoted in some works, and according to what we saw at
the beginning of this section:
P |Mn − mX|2
ε
σ2
X
nε2
(4.7.988)
or:
P (|Mn − mX| ε) 1 −
σ2
X
nε2
(4.7.989)
Therefore for ∀ε 0:
lim
n→+∞
P (|Mn − mX| ε) = 1 (4.7.990)
4.7.11 Characteristic Function
In probability theory and statistics, the characteristic function of any real-valued random vari-
able completely defines its probability distribution. If a random variable admits a probability
density function, we will prove further below that then the characteristic function is the inverse
Fourier transform of the probability density function. Thus it provides the basis of an alternative
route to analytical results compared with working directly with probability density functions or
cumulative distribution functions.
Before giving an engineer proof of the famous central limit theorem, first let us introduce the
concept of characteristic function, which is central in statistics.
First, remember that the Fourier transform is given his in its physicist version (see section
Sequences and Series) by the relation:
F(k) =
1
√
2π
+∞ˆ
−∞
f(x)e−ixk
dx (4.7.991)
Let us recall that the Fourier transform is an analogue of the Fourier series theory for non-
periodic functions, and allows to associate them a frequencies spectrum. To a given factor, it is
a bilateral Laplace transform given by:
L[f(x)] = F(p) =
+∞ˆ
−∞
f(x)e−px
dx (4.7.992)
where p is the complex variable given in the present case by (the real part is zero, because the
Fourier transform is the particular case of a Laplace transform whose real part of the variable
is zero: then make Fourier transform is like make a Laplace transform on the axis of imaginary
numbers only):
p = ik (4.7.993)
Now we want to prove that if:
g(x) = f (x) ⇒ G(k) = ikF(k) (4.7.994)
In other words, we are looking for a simplified expression of the Fourier transform of the deriva-
tive of f(x).

Draft
Proof 4.27.9. We start from:
G(k) =
1
√
2π
+∞ˆ
−∞
f (x)e−ixk
dx (4.7.995)
An integration by parts gives (see section Differential and Integral Calculus):
+∞ˆ
−∞
f (x)e−ixk
dx = f(x)e−ixk x=+∞
x=−∞
+ ik
+∞ˆ
−∞
f(x)e−ixk
dx (4.7.996)
By imposing that f tends to zero at inﬁnity, then we have:
f(x)e−ixk x=+∞
x=−∞
= 0 (4.7.997)
and:
G(k) =
1
√
2π
+∞ˆ
−∞
f (x)e−ixk
dx =
ik
√
2π
+∞ˆ
−∞
f(x)e−ixk
dx = ikF(k) (4.7.998)
Q.E.D.
This is the ﬁrst result we needed.
Now let us prove that if:
g(x) = xf(x) ⇒ G(k) = iF (k) (4.7.999)
Proof 4.27.10. So we start from:
G(k) =
1
√
2π
+∞ˆ
−∞
xf(x)e−ixk
dx = i
d
dk

 1
√
2π
+∞ˆ
−∞
f(x)e−ixk
dx

 = iF (k)
= i
1
√
2π
+∞ˆ
−∞
f(x)
d
dk
e−ixk
dx = i
1
√
2π
+∞ˆ
−∞
f(x)(−ix)e−ixk
dx =
1
√
2π
+∞ˆ
−∞
xf(x)e−ixk
dx
(4.7.1000)
Q.E.D.
This is the second result we needed.
Now let us perform the calculation of the Fourier transform of the Normal centered-reduced
distribution (this choice is not innocent...):
y = fX(x) =
1
√
2π
e−x2/2
(4.7.1001)

Draft
We know that this latter relation is trivially solution of the following differential equation (or it
satisﬁes it in other words...):
y = −xy (4.7.1002)
taking the Fourier transform of both sides of the equality, we use the previous two results:
g(x) = f (x) then G(k) = ikF(k) (4.7.1003)
g(x) = xf(x) then G(k) = iF (k) (4.7.1004)
We have:
ikF(k) = −iF (k) (4.7.1005)
Or:
−kdk =
1
F(k)
dF(k) (4.7.1006)
Then after integration:
kˆ
0
−kdk =
kˆ
0
1
F(k)
dF(k) ⇒ −
1
2
k2
|k
0 = ln(F(k))|k
0 ⇒ −
1
2
k2
= ln(F(k)) − ln(F(0))
⇒ −
1
2
k2
= ln
F(k)
F(0)
⇒ e
−
1
2
k2
=
F(k)
F(0)
⇒ F(k) = F(0)e
−
1
2
k2
(4.7.1007)
As:
F(k) =
1
√
2π
+∞ˆ
−∞
f(x)e−ixk
dk (4.7.1008)
We therefore have:
F(0) =
1
√
2π
+∞ˆ
−∞
f(x)e−ikx
dx =
1
√
2π
+∞ˆ
−∞
f(x)dx =
1
√
2π
+∞ˆ
−∞
1
√
2π
e−x2/2
dx
=
1
2π
+∞ˆ
−∞
e−x2/2
dx
(4.7.1009)
We proved during our study of the Normal distribution that:
+∞ˆ
−∞
e−x2/2
dx =
√
2π (4.7.1010)
Therefore:
F(0) =
1
2π
+∞ˆ
−∞
e−x2/2
dx =
1
2π
√
2π =
1
√
2π
(4.7.1011)

Draft
We then have (very important result!):
F(k) =
1
√
2π
e−k2/2
(4.7.1012)
Let us now introduce the characteristic function as deﬁned by statisticians:
φX(ω) = E eikX
=
+∞ˆ
−∞
eikx
fX(x)dx (4.7.1013)
which is an important and powerful analytical tool for analyzing a sum of independent random
variables. In addition, this function contains all the information characteristic of the random
variable X.
Remark
The notation is not innocent since the E[...] represents an expected mean of the density
function with respect to the complex exponential.
Therefore the characteristic function of the Normal reduced centered random variable of distri-
bution law:
fX(x) =
1
√
2π
e−x2/2
(4.7.1014)
becomes easy to determine because:
φX(k) =
+∞ˆ
−∞
eikx
fX(x)dx =
√
2πF(−k) (4.7.1015)
This is why the characteristic function of the reduced centered Normal distribution is often
assimilated to a simple Fourier transform (see section Sequences and Series).
And thanks to the previous result:
F(k) =
1
√
2π
e−k2/2
⇒ F(−k) =
1
√
2π
e−(−k)2/2
=
1
√
2π
e−k2/2
(4.7.1016)
Therefore:
φX(k) =
√
2π
√
2π
e−k2/2
= e−k2/2
(4.7.1017)
which is the result we will need for the central limit theorem that we will study just after. This
characteristic function is equal, to a given constant, to the probability density of the law. Then
we say that the characteristic function of a Gaussian function is Gaussian function....
But before that, let us look a little closer this characteristic function:
φX(ω) = E eikx
=
+∞ˆ
−∞
fX(x)dx (4.7.1018)

Draft
Using a Maclaurin development (see section Sequences and Series) and changing some nota-
tions we get:
φX(ω) =
+∞ˆ
−∞
+∞
k=0
(iω)k
k!
xk
fX(x)dx (4.7.1019)
and by inverting the sum and the integral, we have:
φX(ω) =
+∞
k=0
(iω)k
k!


+∞ˆ
−∞
xk
fX(x)dx

 =
+∞
k=0
(iω)k
k!
Mk
X (4.7.1020)
This characteristic function contains therefore all the moments (general term used for the mean
and variance) of X.
4.7.12 Central Limit Theorem
The central limit theorem is a set of results from the early 20th century on the weak convergence
in probability of a sequence of random variables. Intuitively, from these results, any sum (im-
plicitly: the average of these variable) of independent random variables identically distributed
tends to some given random variable. The best known and most important result is simply
named central limit theorem concerning a sum of independent random variables with existing
variance whose number tends to infinity and it is this that we will prove heuristically here.
In the simplest case, considered below for the proof of the theorem, these variables are contin-
uous, independent and have the same mean and the same variance. To try to get a finite result,
we should center this sum by subtracting its average and reduce it by dividing it by its standard
deviation. Under fairly broad conditions, the probability distribution (of the average) converges
to a Normal centered reduced distribution. The ubiquity of the Normal distribution is explained
by the fact that many considered random phenomena are due to the superposition of an infinity
of many small causes.
This probability theorem has an interpretation in mathematical statistics. The latter associates a
probability distribution to a population. Each element extracted from the population is consid-
ered as a random variable and by bringing together a number n of these supposed independent
variables, we get a sample. The sum of these random variables divided by n gives a new vari-
able named the empirical mean. This, once reduced, tends to a Normal reduced variable as n
approaches infinity as we know.
The central limit theorem tells us what we should expect in terms of sums of independent
random variables. But what about the products? Well, the logarithm of a product (strictly
positive factors) is the sum of logarithms of factors, so that the logarithm of a product of random
variables (with strictly positive values) tends to a Normal distribution, resulting a log-normal
distribution for the product itself (see our study of the Log-Normal distribution earlier above).
In itself, the convergence to the Normal distribution (asymptotic normality) of many random
variables when their number tends to infinity only interests the mathematician. For the practi-
tioner, it is worth stopping shortly before the limit: the sum of a large number of these variables
is nearly Gaussian, which often provides a more usable approximation than the exact law.

Draft
Conversely, we can say that no concrete phenomenon is really Gaussian because it can not
exceed certain limits, especially if its values are all positive (or all negative).
Proof 4.27.11. Given {Xi}i=1...+∞ a suite (sample) of continuous random variables (in our
simplified proof...), independent (independent measures of physical or mechanical phenomena
for example) and identically distributed, for which the average µX and standard deviation σX
exist (this means that the central limit theorem works for finite variance phenomena only as far
as we know!!!).
We saw earlier in this section that:
Zn =
Sn − nµ
σ
√
n
=
Mn − µ
σ/
√
n (4.7.1021)
are the same expressions of centered reduced variable generated using a sequence of n iden-
tically distributed random variables that by construction therefore has a zero mean and unit
variance:
E(Zn) = 0 and V(Zn) = 1 (4.7.1022)
Let us develop the first form of the prior-previous equality (the both are anyway equal!):
Zn =
1
√
nσ
n
i=1
(Xi − µ) =
1
√
nσ
n
i=1
Xi − nµ (4.7.1023)
Now using the characteristic function of the Normal centered-reduced distribution (we simplify
at the same time the notations of the estimators of the average and standard deviation):
φZn (ω) = E[eiωZn
] = E exp iω
n
i=1(Xi − µ)
√
nσ
(4.7.1024)
As the random variables Xi are independent and identically distributed, we get:
φZn (ω) = E exp iω
X1 − µ
√
nσ
· . . . · E exp iω
Xn − µ
√
nσ
= E exp iω
X − µ
√
nσ
n
(4.7.1025)
A Taylor expansion (see section Sequences and Series) the term between braces gives at the
third order (Maclaurin series expansion of the exponential):
E exp iω
X − µ
√
nσ
∼= E 1 + i
ω
√
nσ ¯X
(X − µ) −
ω2
nσ2
(X − µ)2
(4.7.1026)
Finally:

Draft
φZn (ω) ∼= E 1 + i
ω
√
nσ
(X − µ) −
ω2
nσ2
(X − µ)2
n
=


+∞ˆ
−∞
1 + i
ω
√
nσ
(X − µ) −
ω2
nσ2
¯X
(X − µ)2
fX(x)dx


n
=


+∞ˆ
−∞
fX(x)dx +
+∞ˆ
−∞
iXω
√
nσ
fX(x)dx −
+∞ˆ
−∞
iµω
√
nσ
fX(x)dx −
ω2
nσ2
+∞ˆ
−∞
(X − µ)2
fX(x)dx


n
=

1 +
iω
√
nσ ¯X
+∞ˆ
−∞
XfX(x)dx −
iµω
√
nσ ¯X
+∞ˆ
−∞
fX(x)dx −
ω2
nσ2
¯X
+∞ˆ
−∞
(X − µ)2
fX(x)dx


n
=

1 +
iω
√
nσ ¯X
µ −
iµω
√
n
−
ω2
nσ2
+∞ˆ
−∞
(X − µ)2
fX(x)dx


n
=

1 −
ω2
nσ2
+∞ˆ
−∞
(X − µ)2
fX(x)dx


n
= 1 −
ω2
nσ2
¯X
V(X)
n
= 1 −
ω2
2n
n
(4.7.1027)
Let us put:
−
ω2
2n
=
1
x
(4.7.1028)
Then we have:
φZn (ω) ∼= 1 +
1
x
−
ω2
x
2
= 1 +
1
x
x −
ω2
2
(4.7.1029)
So as x approaches inﬁnity (see section Functional Analysis):
φZn (ω) ∼= lim
n→+∞
1 +
1
x
x −
ω2
2
= lim
n→+∞
1 +
1
x
x −
ω2
2
= e
−
ω2
2 (4.7.1030)
We ﬁnd back the characteristic function of the reduced centered Normal distribution!
In two words, the Central Limit Theorem (CLT) said that for large samples, the centered and
reduced sum of n random variables independent ad identically distributed follow a Normal
centered and reduced distribution. And so we have verbatim to the empirical mean:
Mn − µ
σ/
√
n
= N(0, 1) ⇔ Mn = N µ,
σ
√
n
(4.7.1031)
Now we will illustrate the central limit theorem in the case of a sequence {Xi} of discrete
random variables following a Bernoulli distribution with parameter equal to 1/2.

Draft
We can imagine that XN represents the result obtained in the n-th launch of a coin (assigning
the number 1 for head and 0 for tail). Let us write:
¯Xn =
1
n
n
k=1
Xk (4.7.1032)
the average. We obviously have for all n in this special case:
E(Xn) =
1
2
V(Xn) =
1
4
(4.7.1033)
and therefore:
E( ¯Xn) =
n
k=1
1
2
=
1
2
V( ¯Xn) =
1
n2
n
k=1
1
4
=
1
4n
(4.7.1034)
After having centered and reduced ¯Xn we get:
¯Xn − E( ¯Xn)
V( ¯Xn)
= 2
√
n ¯Xn −
1
2
(4.7.1035)
Let us note Φ as always the cumulative Normal reduced centered distribution.
The central limit theorem says that for any t ∈ R:
lim
n→+∞
P 2
√
n ¯Xn −
1
2
≤ t = Φ(t) (4.7.1036)
Using Maple 4.00b we have plot in blue some graphs of the function:
t → Fn(t) = P 2
√
n ¯Xn −
1
2
≤ t (4.7.1037)
for different values of n. We have Represented in red the function Φ.
For n = 1:
Figure 4.108 – First approach of the Bernoulli distribution by the Normal distribution according to the CLT
For n = 2:

Draft
Figure 4.109 – Second approach of the Bernoulli distribution by the Normal distribution according to the CLT
For n = 5:
Figure 4.110 – Fifth approach of the Bernoulli distribution by the Normal distribution according to the CLT
For n = 30:

Draft
Figure 4.111 – Thirteenth approach of the Bernoulli distribution by the Normal distribution according to the CLT
These graphs were obtained with Maple 4.00b using the following commands:
bwith@st—tsAX
bwith@plotsAX
beIXaplot@re—viside@tCIABst—tev—lf‘d™dfD˜inomi—ld‘IDHFS““@trun™@@tCIAGPAADtaEPFFP
DyaHFFID™olora˜lueAX
bePXaplot@re—viside@tCsqrt@PAABst—tev—lf‘d™dfD˜inomi—ld‘PDHFS““
@trun™@@tBsqrt@PACPAGPAADtaEsqrt@PAEIFFsqrt@PACIDyaHFFID™olora˜lueAX
beQXaplot@re—viside@tCsqrt@SAABst—tev—lf‘d™dfD˜inomi—ld‘SDHFS““
@trun™@@tBsqrt@SACSAGPAADtaEsqrt@SAEIFFsqrt@SACIDyaHFFID™olora˜lueAX
beRXaplot@st—tev—lf‘™dfDnorm—ld“@tADtaESFFSAX
beSXaplot@re—viside@tCsqrt@QHAABst—tev—lf‘d™dfD˜inomi—ld‘QHDHFS““
@trun™@@tBsqrt@QHACQHAGPAADtaEsqrt@QHAEIFFsqrt@QHACIDyaHFFID™olora˜lueAX
bdispl—y@eIDeRAY
bdispl—y@ePDeRAY
bdispl—y@eRDeQAY
bdispl—y@eSDeRAY
clearly show the convergence of Fn to Φ.
In fact we see that convergence is downright uniform which is conﬁrmed by the Moivre-
Laplace central limit theorem:
Given Xn a sequence of independent random variables with the same Bernoulli parameter p,
0 p 1. Therefore:
Fn(t) = P 2
√
n ¯Xn −
1
2
≤ t = Φ(t) (4.7.1038)
tends uniformly to Φ(t) on R when n → +∞.

Draft
4.7.13 Univariate Hypothesis and Adequation tests
During our study of confidence intervals, remember that we get the below few relations (it is
only a sample of the most important one proved above!):
¯X − Zα/2
σ
√
n
≤ µ ≤ ¯X + Zα/2
σ
√
n
(4.7.1039)
and:
nS2
∗
χ2
α/2(n)
≤ σ2
≤
nS2
∗
χ2
1−α/2(n)
(4.7.1040)
and:
(n − 1)S2
χ2
α/2(n − 1)
≤ σ2
≤
(n − 1)S2
χ2
1−α/2(n − 1)
(4.7.1041)
and finally:
¯X −
S
√
n
Tα/2(n − 1) ≤ µ ≤ ¯X +
S
√
n
Tα/2(n − 1) (4.7.1042)
that allowed to do statistical inference based on the knowledge or not of the true mean or vari-
ance of the whole or on a sample of the population. In other words, under in what range stood
a given moment (mean or variance) as a function of the chosen confidence level α. We had
seen that the second interval above can only be hardly used in practice (assumes known the
theoretical average) and therefore we prefer the third.
We will also prove in details later the following two intervals:
1
Fα/2(n, m)
S2
∗X
S2
∗Y
≤
σ2
X
σ2
Y
≤
1
F1−α/2(n, m)
S2
∗X
S2
∗Y
(4.7.1043)
and:
1
Fα/2(n − 1, m − 1)
S2
X
S2
Y
≤
σ2
X
σ2
Y
≤
1
F1−α/2(n − 1, m − 1)
S2
∗X
S2
∗Y
(4.7.1044)
The first interval above can be also used with difficulty in practice (assumes the theoretical
average as know) and therefore we prefer the second.
Definition: When we want to know if we can trust the value of a statistic (mean, median,
variance, correlation coefficient, etc.) with some certainty, we speak of hypothesis testing and
especially of conformity test (we speak of adequation test when the purpose is to check that
measures will follow a particular distribution law).
Remark
The reader must also remember, as already said earlier in this section, that we have put
many other confidence interval techniques detailed proofs related for example for regres-
sion techniques in the section of Theoretical Computing.

Draft
Hypothesis testing are intended to check whether the sample can be considered as extracted
from a given population or be representative of this population vis-a-vis of parameter such as
a mean, variance or the observed frequency. This implies that the theoretical distribution of
the parameter is known at the population level. The hypothesis tests are not made to prove the
null hypothesis (usually expressing equality or uniformity between different populations), but
to eventually reject it (to be exact the rejection of the null hypothesis is more robust). In terms
of the communication of statistical tests a number of experts recommend:
1. To always communicate the p-value with four decimal places (we will come back later on
this concept).
2. Never say that a low p-value shows a significant magnitude of the effect studied because
it is not necessary true (to check you just take a phenomenon of very small amplitude on
a large sample and then the p-value will become very small by construction). Once again
will discuss this more deeply later.
3. To always give the confidence interval of the test whether it is unilateral or bilateral.
4. To be careful to not to set a rejection threshold to the test except if a standard (norm) or
legislation requires it (in which case we will specify which one).
5. To never say that the test is proved or significant or even statistically significant.
Just say that the result is statistical or that we have the likelihood of the data knowing
the null hypothesis and that’s it!
6. If the interest is to show the null hypothesis and that it is not rejected, since often its
statistical power is low, you will need to repeat the experience to reinforce the conclusion.
7. If the interest is to reject the null hypothesis and that this is true, a good scientific practice
is to look for additional studies that would faulty conclusion.
8. If there is for example no statistical difference between two values, therefore this does
not mean that there is presence of statistical equivalence. It is then necessary to proceed
to equivalence tests.
9. The rejection of the null hypothesis does not mean that the mechanism of the phenomenon
has been highlighted but just indicates, for refresh, an information about the size of the
dataset a posteriori.
10. We communicate the posteriori power of the test.
To resume, the studies must be communicated respecting the principle of truthfulness, after
having been the subject of appropriate controls, and must be exposed, described and presented
with impartially (some people say a-theoric). We must not confuse between objectives and
speculatives results. The conclusions should be the most faithful expression of the content of
facts and datas.
For example, if we want to know with some confidence whether a given mean of a population
sample is realistic about the true unknown theoretical mean we will use the Z-test which is

Draft
simply:
¯X − Zα/2
σ
√
n
≤ µ ≤ ¯X + Zα/2
σ
√
n
(4.7.1045)
Now remember that we have proved that if we have two random variables of law:
N(µ1, σ2
1) N(µ2, σ2
2) (4.7.1046)
then subtraction (differentiate) the averages gives:
N(µ2 − µ1, σ2
1 + σ2
2) (4.7.1047)
So for the difference of two average of random variables from two independent population
samples we obtain directly:
N

 ¯X2 − ¯X1,
σ2
1
n1
+
σ2
2
n2

 (4.7.1048)
We can then adapt the Z − test as:
Zcalc
=
¯X − µ
σ/
√
n
=
¯X − µ
σ2
n
=
( ¯X2 − ¯X1) − (µ2 − µ1)
σ2
1
n1
+
σ2
2
n2
(4.7.1049)
¯X2 − ¯X1 − Z
σ2
1
n1
+
σ2
2
n2
µ2 − µ1
¯X2 − ¯X1 + Z
σ2
1
n1
+
σ2
2
n2
(4.7.1050)
The relation that is very useful when for two samples of two populations, we want check if there
is a statistically significant difference of the mean to a given fixed level of confidence level α
and the associated probability for this difference: Therefore:
|Zcalc
| |Zα/2| (4.7.1051)
U/k
V/l
= Fk,l(x) =
Γ
k + 1
2
k
l
k
2
Γ
k
2
Γ
l
2
x
k
2
−1
1 +
kx
l
−k+l
2
(4.7.1052)
We the speak of two-sample Z-test and it is used lost in the industry to ensure the equality of
two averages of two measurements of populations when standard deviation is known.
And if the theoretical standard deviation is not known, we use the Student T-test (used a lot
in pharmacoeconomics) proved above:
¯X −
S
√
n
Tα/2(n − 1) ≤ µ ≤ ¯X +
S
√
n
Tα/2(n − 1) (4.7.1053)

Draft
In the same idea for standard deviation, we use the Chi-square (variance) test as already
proved above:
(n − 1)S2
χ2
α/2(n − 1)
≤ σ2
≤
(n − 1)S2
χ2
1−α/2(n − 1)
(4.7.1054)
And when we want to test for the equality of variance of two populations we use Fisher F-test
(that will be proved below during our study of the analysis of variance):
1
Fα/2(n − 1, m − 1)
S2
X
S2
Y
≤
σ2
X
σ2
Y
≤
1
F1−α/2(n − 1, m − 1)
S2
∗X
S2
∗Y
(4.7.1055)
In practice we must be aware that the purpose of a test is very often to show that the effect
is significant. It is then customary to say that the test is successful if the null hypothesis is
rejected in favor of the alternative hypothesis! When the practitioner knows that the effect is
significant and his test fails to reject the null hypothesis this is sometimes named the dilemma
of not rejecting the null hypothesis. As we will see a little further, the idea is then to calculate
an a posteriori test power (it is then named by some software like SPSS, observed power)
and adapt the sample size accordingly to have an acceptable power according to tradition.
4.7.13.1 Direction of hypothesis test
The fact that we get all values satisfying a right bounded test AND (!) left bounded is what we
name in the general case a two-tailed test as it includes the left sided and right sided unilateral
tests. Thus, all the above tests are in a bilateral form, but we could make a unilateral use too!
We use one-tailed test when the expected difference (or difference to highlight) can only go in
one direction (typically in the case of clinical trials or during a corrective quality action in the
industry for which we expect a improvement going in one single direction). Unilateral tests are
sometimes named non-inferiority tests (left-sided) or non-superiority test (right-sided).
Below we presented for example a right unilateral test (since the rejection region is on the right
and therefore the cumulative probability is left-sided) and a two-sided test:
Figure 4.112 – Illustration of a test (or confidence level) unilateral right and bilateral
We can also summarize how to determine the p-value (which will be discussed more in detail
further below) with the following diagram:

Draft
Figure 4.113 – Resume ﬁgure to determine the p-value of parametrical test with symmetrical distributions.
Deﬁnition: The p-value is the probability of obtaining an effect at least as extreme as the one
in your sample data, assuming the truth of the null hypothesis. In other words, if the null
hypothesis is true, the p-value is the probability of obtaining your sample data. It answers the
question, are your sample data unusual if the null hypothesis is true? If you are thinking that
the p-value is the probability that the null hypothesis is true, the probability that you are making
a mistake if you reject the null, or anything else along these lines, this is the most common
misunderstanding of practitioners.
The common misconception is what we’d really like to know. We would loooove to know the
probability that a hypothesis is correct, or the probability that we’re making a mistake. What
we get instead is the probability of our observation, which just isn’t as useful.
It would be great if we could take evidence solely from a sample and determine the probability
that the sample is wrong. Unfortunately, that’s not possible and this for logical reasons when
you think about it. Without outside information, a sample can’t tell you whether it’s represen-
tative of the population.
The p-values are based exclusively on information contained within a sample. Consequently,
p-values can’t answer the question that we most want answered, but there seems to be an irre-
sistible temptation towards interpreting it that way.
Let us also note that the hypothesis tests on the standard deviation (variance), the mean or
correlation are named parametric tests in reverse of the non-parametric tests that we will see
much further below.

Draft
Remarks
R1. There is also another deﬁnition of the concept of parametric and non-parametric
tests that we will se later below (a little different because more precise).
R2. Warning! Some authors or teachers sometimes talk about left-sided for a right-
sided test... In fact it is simply a choice of vocabulary. If the reference of teaching is
not the rejection area but the acceptance area, then it is clear that the right and left are
concepts that are reversed...
Finally, many software calculates what we name therefore the p-value which is limit calcu-
lated risk (probability threshold) α that might have set the statistician to be at the limit between
acceptance of the null hypothesis and its rejection (remember that a successful test does not
prove anything and a rejection is better but still do not prove anything!). So the p-value is a
fundamental value in statistics because it gives the possibility to quantify the likelihood of the
null hypothesis H0 (acceptance or rejection).
But strictly speaking the p-value is the conditional (Bayesian) probability, that our data satisfy
the null hypothesis P(data|H0) and not the probability of the null hypothesis knowing the data!
While the difference may be small as we saw in the section Probabilities, it is not zero! So in
reality the p-value says nothing about the hypothesis itself, but it provides information on the
experimental data.
For hypothesis testing, for example, the 5% risk threshold α is the risk to reject the null hypoth-
esis even though it is true. If the risk imposed/chosen is 5% and the calculated p-value is less
(in most tests but be careful because this is not a generality!!!), the test fails (rejection of the
null hypotheses) in favor of an alternative hypothesis denoted by H1 or Ha. Never forget that
reject the test is always better in term of power than accepting it.
The alternative hypothesis has of course itself has its own risk that we denote by β and its
own p-value. So when the null hypothesis is not rejected, the risk associated with this decision
is a risk of the second kind type. To assess this, we should calculate the power of the test
considered (see proofs later below).
Perhaps, to better understand, here is an illustration of a particular case of an bilateral hypothesis
test of the average for a typical random variable following a Normal distribution (basically it’s
almost the same principle for all tests...):

Draft
Figure 4.114 – Null and alternative hypothesis of a special case of two-sided test
Thus, in the case presented above, we see better why the null hypothesis can be accepted or
rejected in favor of the alternative hypothesis (which is of the same law that the null hypothesis
but just shifted) depending on the reference value measured that will be used for the test (in the
special case of the above figure it is the arithmetic mean of measurements).
We also note that the red area of the alternative hypothesis, corresponding to the cumulated
probability β, is partially merged with the yellow part of the null hypothesis. This is why we
can sometimes accept the null hypothesis wrongly. However, we see that smaller is β, more
the alternative hypothesis would be far from the red boundary zone of the null hypothesis (this
would correspond to a translation to the right in this case) and less the likelihood of a false
conclusion is big. This is why we talk about risk β because smaller it is, the better. Verbatim
more 1 − β is big, then smaller is the risk of confusing the null and alternative hypothesis. This
is why 1 − β is named power of the test (see below the subsection that is devoted to this
concept).
We accept (sadly do not reject!) the null hypothesis if the p-value is greater than 5% (0.05). In
fact, bigger is the p-value is, the better it is because for people interested to the null hypothesis
because the confidence interval is becoming smaller. If the confidence interval has to be huge
(very close to 100%) for the p-value is very small so the analysis does not really make sense
physically speaking in the point of view of the null hypothesis!
Thus, if the p-value is low, that mean we should take a low risk α of error, therefore accept in

Draft
almost all cases the tested hypothesis (H0)...
Remarks
R1. We should never say that we accept a hypothesis or that it is true or false as
these terms are too strong and might suggest a scientific proof. We should say if we
reject or does not reject the null hypothesis and it is possibly correct or incorrect.
R2. For bilateral test hypotheses, we can for example say that we have (or not have) a
significant difference between the measured reference value and the expected value. For
one-sided tailed, we can say that the measured reference value is significantly bigger or
smaller than the expected value.
R3. Moreover if the reader has well understand the construction of hypothesis testing,
the fact of wrongly reject a hypothesis (Type I error or Error of the first kind) is more
robust than accept it wrongly (Type II error or Error of the second kind).
R4. The reader will also have noticed with the help of the previous figure that a one sided
test has a higher power than a one-sided test (for a same risk threshold of course!). Thus,
a statistically insignificant difference in bilateral test, can be statistically significant in a
one sided test.
R5. If the p-value is close but not to the limit threshold (rejection) value we say that the
effect is marginally significant.
A big issue with hypothesis test is what we name p-hacking: the underlying idea consists of
replicating a test dozens of times until it provides the conclusion that the experimenter like... or
try all possible combinations of variables until we found something significant or even just take
big samples (as we know that NHST tests fail systemically when samples are big).
Definitions:
D1. The probability α of Type I error (error of the first kind / false negative) is the probability
of rejecting the null hypothesis when it is true.
D2. The probability β of Type II error (error of the second kind / false positive) is the prob-
ability of maintaining the null hypothesis when it is false. Since, by definition, power is
equal to one 1 − β, the power of a test will get smaller as beta gets bigger.
Thus a traditional criteria selection is to use the following principle: among all the tests that
have the same size of type I error, choose one that has the smallest size of the value of the Type
II error.
In general, the magnitude of the Type II error increases when that of the Type I error decreases.
We can not, as far as we know, minimize the two errors at the same time. For this reason, we
often take a fixed value of α, size of the type I error, and we minimize β, magnitude of the Type
II error (i.e. we increase the power 1 − β.

Draft
To close this subject, here are the three types of situations testing hypotheses on the statistics
that is the average in the framework of an underlying Normal distribution and whose mean
is in this particular case zero and of unit variance (because we can often come back to this
particular case by centering and reducing the underlying random variable or by doing special
transformations):
Figure 4.115 – The three possible scenarios for a hypothesis test on the average
Let us indicate that it makes no sense (as opposed to what we can sometimes read in some paper

Draft
or electronic media) to have the following null hypotheses in the special case shown above:
H0 : µ ≤ 0
H0 : µ ≥ 0
H0 : µ = 0
(4.7.1056)
with the alternative hypothesis that automatically follows (we did not write it because it is
useless and obvious). The reason is simple: how could you position your reduced centered
Normal distribution if the mean is not ﬁxed ... ??? This is why the null hypothesis in the context
of the tests on the mean (and some other tests) are always written with an equality!
To summarize, we can say that if we make a decision, we can make an error and it is best not
to make mistakes often. Clearly, the probability of saying something stupid must be known and
preferably small.
4.7.13.1.1 Simpson’s Paradox (sophism)
Simpson’s paradox (in fact it’s a sophism and not a paradox because it is just a special
case of omitted variable bias), is a paradox in probability and statistics, in which a trend
appears in different groups of data but disappears or reverses when these groups are combined.
This result is often encountered especially in social-science (HR; Marketing, Psychology), and
medical-science statistics and is particularly confounding when frequency data are unduly given
causal interpretations. Simpson’s paradox disappears when causal relations are brought into
consideration.
Let us see an famous real example for a real medical study:
The table below shows the success rates and numbers of treatments for treatments involving
both small and large kidney stones, where Treatment A includes all open surgical procedures
and Treatment B is percutaneous nephrolithotomy (which involves only a small puncture). The
numbers in parentheses indicate the number of success cases over the total size of the group (for
example, 93% equals 81 divided by 87):
Treatment A Treatment B
Small stones Group 1
93%(81/87)
Group 2
87%(234/270)
Large stones Group 3
73%(192/263)
Group 4
69%(55/88)
Both 78%(273/350) 83%(289/350)
The paradoxical conclusion is that treatmentA is more effective when used on small stones, and
also when used on large stones, yet treatment B is more effective when considering both sizes
at the same time. In this example the lurking variable (or confounding variable) of the stone
size was not previously known to be important until its effects were included.
Formally, if we denote (we change the notations!!!!) A the result, B the treatment, and C the
stones, in the general case, with the Simpson paradox it is possible to have using Bayesian

Draft
probabilities:
P(A|B, C) P(A|Bc
, C) and P(A|B, Cc
) P(A|Bc
Cc
) yet also: P(A|B) P(A|Bc
)
(4.7.1057)
Which treatment is considered better is determined by an inequality between two ratios (suc-
cesses/total). The reversal of the inequality between the ratios, which creates Simpson’s para-
dox, happens because two effects occur together:
1. The sizes of the groups, which are combined when the lurking variable is ignored, are
very different. Doctors tend to give the severe cases (large stones) the better treatment
A, and the milder cases (small stones) the inferior treatment B. Therefore, the totals are
dominated by groups 3 and 2, and not by the two much smaller groups 1 and 4.
2. The lurking variable has a large effect on the ratios, i.e. the success rate is more strongly
inﬂuenced by the severity of the case than by the choice of treatment. Therefore, the
group of patients with large stones using treatment A (group 3) does worse than the group
with small stones, even if the latter used the inferior treatment B (group 2).
Based on these effects, the paradoxical result is seen to arise by suppression of the causal effect
of stone size on successful treatment. The paradoxical result can be rephrased more accurately
as follows: When the less effective treatment (B) is applied more frequently to easier cases, it
can appear to be a more effective treatment.
Simpson’s paradox usually fools us on tests of performance. In a famous example, researchers
concluded that a newer treatment for kidney stones was more effective than traditional surgery,
but it was later revealed that the newer treatment was more often being used on small kidney
stones. More recently, on elementary school tests, minority students in Texas outperform their
peers in Wisconsin, but Texas has so many minority students that Wisconsin beats it in state
rankings. It would be a shame if Simpson’s paradox led doctors to prescribe ineffective treat-
ments or Texas schools to waste money copying Wisconsin.
Consider another funny illustrated example in Marketing! Consider we have the following
information:
Figure 4.116 – Marketing Campaign performance

Draft
Obviously the campaign B perform the best into coveting an e-mail to click!
But now splitting in clicking high valuable links (costly products: high ticket conversion) or
low valuable links (low ticket conversion) we get:
Figure 4.117 – Marketing Campaign performance in ticket conversion covariate
4.7.13.2 Power of a test
When the effect is actually large, we can expect that we need less observations to demonstrate
it that when the effect is small ... but how much exactly? Did we have the possibility to do this,
in terms of number of measures, to prove what we seek? Should we go about it differently
and change the device to do the observation/experiment?
To study in more detail the concept of power of a test that we have so far only mentioned,
remember the following ﬁgure that we have already see just a little bit earlier:

Draft
Figure 4.118 – Null and alternative hypothesis of a special case of two-sided test
In the particular example above, we will obviously reject the null hypothesis H0 if ¯X 1.96 or
¯X 1.96. Imagine that under the alternative hypothesis Ha, if we measured 2.5 for ¯X, we will
have for power of the test:
1 − β = 1 − P YN(2.5,1) ≤ 1.96 = 1 − P YN( ¯X,1) ≤ Z1−α/2 = 1 − 0.2946 ∼= 0.705
(4.7.1058)
So the test is relatively strong/powerful (in practice, we consider a test to be powerful if its value
is above of 80% - that is to say the β ≤ 20%). Thus, we see that the (a posteriori!) power 1−β is
even larger than the p-value is small (and respectively the power will be a posteriori even smaller
than the p-value is great). Therefore the a posteriorii power is in decreasing correspondence
with the p-value (in practice it is however a bit absurd to make these calculations a posteriori
but some scientiﬁc paper require this information).

Draft
4.7.13.3 Power of the one sample Z-test
Very generally, in the case of a bilateral test, the above relation will be written:
1 − β = 1 − P YN( ¯X,1) ≤ µ + Z1−α/2 = 1 − P YN(0,1) ≤ µ − ¯X + Z1−α/2 (4.7.1059)
If the standard deviation of the mean (e.g. variance) is not unitary, we have:
1 − β = 1 − P YN(0,σ ¯X ) ≤ µ − ¯X + Z1−α/2σ ¯X (4.7.1060)
Therefore we get:
1 − β = 1 − P YN(0,1) ≤
µ − ¯X
σ ¯X
+ Z1−α/2 = 1 − P YN(0,1) ≤
µ − ¯X
σ/
√
n
+ Z1−α/2
(4.7.1061)
written in another way:
1 − β = 1 − P YN(0,1) ≤
µ − ¯X
σ
√
n + Z1−α/2 = 1 − P YN(0,1) ≤ dσ
√
n + Z1−α/2
= P YN(0,1) ≤ −Z1−α/2 − dσ
√
n = P YN(0,1) ≤ Zα/2 − dσ
√
n
(4.7.1062)
It is in this form that we find the power of a bilateral test of the average (power of a 1 sample
Z-test):
1 − β = P YN(0,1) ≤ Zα/2 − dσ
√
n (4.7.1063)
where dσ is sometimes named the size effect and therefore defined by:
dσ =
µ − ¯X
σ
=
δ
σ
(4.7.1064)
and δ is named the difference!
It goes without saying that if the true variance is not known, we must be replace the Normal law
by the Student law as:
1 − β = P YN(0,1) ≤ Tα/2 − dS
√
n (4.7.1065)
with:
dS =
µ − ¯X
S
=
δ
S
(4.7.1066)
Remark
Caution wit a small recurring trap! The development above corresponds to a δ which is
negative with respect to the first example! The relation is a little bit different in the case
where δ is positive but it does not matter because the power of the test is the same in
absolute value!

Draft
For the sample size it quite simple. We have by equating the quantiles:
Z1−β = Zα/2 − dσ
√
n (4.7.1067)
And therefore in bilateral:
n =
Zα/2 − Z1−β
dσ
√
n
2
(4.7.1068)
where we see that if the power of the test is imposed as being equal to 50%, with having
therefore Z1−β that is 0 then we fall back (!) on the relation of the sample size for the Normal
distribution proved earlier above:
n =
Zα/2
dσ
√
n
2
(4.7.1069)
Note also that we sometimes find in the literature the prior-previous relation as follows:
n =
σ2
Z1−α/2 + Z1−β
2
(µ0 − µA)2
(4.7.1070)
Obviously we can set other parameters to determine the value of the remaining variable. We
could also, for example, look for the value of the power of the test by fixing the standard
deviation, the sample size and the risk threshold, etc.
One reader offered us a very elegant way of finding the same result with much less develop-
ments... Indeed, you need only to see in the previous figure that we have:
Zβ
σ
√
n
+ ¯X = Z1−α/2
σ
√
n
+ µ (4.7.1071)
equality from which we draw immediately an equivalent relations to the two previous one
(which obviously gives the same numerical result):
n =
σ Zβ + Z1−α/2
(µ − ¯X)
2
(4.7.1072)
Remark
The attentive reader will have perhaps noticed that we assumed in the previous devel-
opments that the standard deviation of the mean of the null hypothesis and that of the
alternative one is implicitly assumed to be the same... In practice this is almost all the
time so, this is why almost all statistical software require only one standard deviation to
calculate the power of the 1-Sample Z Test. However, for some rare academic softwares,
they ask the standard deviation of the two means. But then the above mathematical de-
velopments are obviously different.
A test power analysis may have several facets:
1. We know the level of the test, the sample size and effect size (implicitly the difference)
and we seek to calculate its power. This allows us to see if our experimental device is
properly calibrated.

Draft
2. We know the desired test power, the risk threshold level and effect size to detect. We seek
then to calculate the sample size needed to mount an effective experimental design.
3. We know the desired test power, the risk threshold level and sample size and we seek to
know the effect size we can hope to highlight.
Without exception, we consider as unnecessary to show a test if the expected power is less than
80%. This power corresponds to a 80% probability of not rejecting the null hypothesis wrongly
or, what remains to the same: a 20% of Type II error.
Obviously, it is possible to make the same reasoning (analytically when possible, otherwise
numerically) with absolutely ALL the hypothesis tests that we have seen until now. So as there
is a little more than a hundred hypothesis testing in the field of statistics as we have already
mentioned... it is obvious that we’re not going to have.... fun ... to make the same developments
to determine the sample size, effect size and power for all these tests but only for the classics
one. As long as we have computers at our disposal with the algorithms integrated by computer
scientists, we do not need to redo any developments that would not bring much. Moreover, the
majority of statistical softwares can calculate usually the power of only 5 to 10 common tests.
Remark
We will not dealing with parametric statistical tests for the detection of outliers in this
book as the Q-Dixon’s test or Grubb’s test and this just because they have too empirical
origin and that they have no interest analytically speaking. By cons, if some readers insist,
and that we have the time, we can put the details on these tests with detailed algorithms
for calculating critical values using a simple spreadsheet and Monte Carlo technique with
any distribution of their choice (but not only for data following a Normal law contrary to
what is written in most books).
4.7.13.4 Power of the one and two samples P-test
Same as with the confidence interval of the Normal distribution with known theoretical standard
deviation (that is to say the entire population), we can determine the number of individuals
(sample size) for a fixed test power for the 1 sample proportion test studied in detail earlier. For
this, we use the same technique as for the power of the one sample Z-test. We write so at first:
Zβ
ˆpˆq
n
+ ˆp = Z1−α/2
pq
n
+ p (4.7.1073)
Hence we deduce:
n =
Zβ
√
ˆpˆq − Z1−α/2
√
pq
p − ˆp
2
(4.7.1074)
So if the power is 50
n =
Z2
1−α/2pq
δ2
(4.7.1075)

Draft
For the power of the test of the difference of two proportions (two sample proportion test), with
the objective of determining the sample size, we have to put n = n1 = n2. Therefore the
developments we get during our study of the test of the difference of two proportions can be
written:
Z =
ˆp1 − ˆp2
ˆp(1 − ˆp)
1
n1
−
1
n2
=
ˆp1 − ˆp2
2ˆp(1 − ˆp)
n
=
ˆp1 − ˆp2
2ˆpˆq
n
(4.7.1076)
with therefore:
ˆp =
n1 ˆp1 + n2 ˆp2
n1 + n2
=
n( ˆp1 + ˆp2)
2n
=
ˆp1 + ˆp2
2
(4.7.1077)
In the same way that we did for the Z-test and the one sample p-test, we have:
Zβ V(ˆp2 − ˆp1 + (ˆp2 − ˆp1) = Z1−α/2
2ˆpˆq
n
+
ˆp1 + ˆp2
2
(4.7.1078)
Therefore:
Zβ V(ˆp2 − ˆp1 + (ˆp2 − ˆp1) = Z1−α/2
2ˆpˆq
n
+ ˆp (4.7.1079)
So what it takes us to assume that the real difference between the two proportions is the average
(which is debatable...).
But we also have (as the samples are independent and using the property of the variance):
Zβ V(ˆp2) + V(ˆp1) + (ˆp2 − ˆp1) = Z1−α/2
2ˆpˆq
n
+ ˆp (4.7.1080)
Therefore:
Zβ
ˆp1 ˆq1
n
+
ˆp2 ˆq2
n
+ (ˆp2 − ˆp1) = Z1−α/2
2ˆpˆq
n
+ ˆp (4.7.1081)
That gives obviously:
Zβ
ˆp1 ˆq1 + ˆp2 ˆq2
n
+ (ˆp2 − ˆp1) = Z1−α/2
2ˆpˆq
n
+ ˆp (4.7.1082)
We have then after rearrangement:
n =
Zβ
√
ˆp1 ˆq1 + ˆp2 ˆq2 − Z1−α/2
√
ˆpˆq
ˆp − (ˆp2 − ˆp1)
2
(4.7.1083)

Draft
4.7.13.5 Analysis of Variance with one fixed factor
The objective of the analysis of variance (at the contrary to what its name might suggest...) is a
statistical technique for comparing the means of two or more populations and that is very widely
used in the pharmaceutical field or in the RD labs or benches test. This method, however, takes
its name from that it use measures of variance to determine the statistical significance, or not,
of the differences of measured averages on populations or samples.
More precisely, the real meaning is to know if the fact that the sample averages are (slightly)
different can be assigned at random sampling or due to the fact that a variability factor actually
generates significantly different samples (if we have the population size, we have obviously not
such question!).
Remark
For more information about the vocabulary and application, the engineer and the re-
searcher should refer to standard ISO 3534-3:1999.
For the analysis of variance named also one factor ANOVA (Analysis Of VAriance) or one
factor ANAVAR (ANAlysis of VARiance), or one-way ANOVA or more rigorously one
fixed factor ANOVA with repetitions or ANOVA with one fixed categorical variable with
repetition, we first recall, as we proved, that the Fisher-Snedecor law is given by the ratio of
two independent random variables that follow a Chi-square law and divided by the degree of
freedom such as:
U/k
V/l
= Fk,l(x) =
Γ
k + l
2
k
l
k
2
Γ
k
2
Γ
l
2
x
k
2
−1
1 +
k
l
x
k + l
2
(4.7.1084)
and we will now see its importance.
Remark
When a factor can have a very large number of levels we consider having chosen some
given level of the factor among a multitude of possible as a random selection. This is why
we speak then in this latter case of random factor which is the subject specific ANOVA
that we will study once those on fixed factors master (e.g. ANOVA mixing fixed and
random factors factors are named mixed ANOVA) .
Let us consider a random sample of size n, say X1, X2, ..., Xn sampled from the law N(µX, σX)
and a random sample of size m, say Y1, Y2, ..., Ym sampled from the law N(µY , σY ) .
Consider now the maximum likelihood estimators of the standard deviation of the Normal law

Draft
traditionally noted in the field of analysis of variance:
S2
∗X =
1
n
n
i=1
(Xi − µX)2
and S2
∗Y =
1
n
n
i=1
(Yi − µY )2
(4.7.1085)
The above statistics are those that we could use to estimate the variances if the average theo-
retical µX, µY were known. So we can use a result proved above in our study of confidence
intervals:
χ2
(n) =
nS2
∗X
σ2
X
and χ2
(m) =
mS2
∗Y
σ2
Y
(4.7.1086)
As the Xi are independent of the Yj (hypothesis that suppose the covariance to be zero, the
converse being for reminder not always!), the random variables:
nS2
∗X
σ2
X
and
mS2
∗Y
σ2
Y
(4.7.1087)
are independent from each other.
Remark
There is an existing type of ANOVA for the case where the variables are not independent
(we then speak about covariate). This is the ANCOVA which means ANalysis of
COvariance and VAriance that uses a mix of linear regression (see section Numerical
Methods) and ANOVA. The purpose of the ANCOVA is to statistically eliminate the
indirect effect of the covariate. We will see much later how it works in details.
We can therefore apply the Fisher-Snedecor law with:
U =
nS2
∗X
σ2
X
; k = n and V =
mS2
∗Y
σ2
Y
; l = m (4.7.1088)
Therefore we get:
U/k
V/l
=
nS2
∗X
σ2
X
/n
mS2
∗Y
σ2
Y
/m
=
S2
∗X
σ2
X
S2
∗Y
σ2
Y
=
S2
∗Xσ2
Y
S2
∗Y σ2
X
= Fn,m(x) (4.7.1089)
Finally:
σ2
X
σ2
Y
=
1
Fn,m(x)
S2
∗X
S2
∗Y
(4.7.1090)
This theorem allows us to deduce the confidence interval of the ratio of two variances when
the theoretical mean is known. Since Fisher function is not symmetrical, the only possibility to
make the inference is to use numerical calculation and then we will note for a given confidence
interval the test as follows:
1
Fα/2(n, m)
S2
∗X
S2
∗Y
σ2
X
σ2
Y
1
F1−α/2(n, m)
S2
∗X
S2
∗Y
(4.7.1091)

Draft
In the case where the averages µX, µY are unknow, we use the unbiased estimators of the vari-
ances noted traditionally in the field of analysis of variance:
S2
X =
1
n − 1
n
i=1
(Xi − ¯X)2
and S2
Y =
1
m − 1
m
j=1
(Yj − ¯Y )2
(4.7.1092)
To estimate the theoretical variances, we use the result proved above:
(n − 1)S2
X
σ2
X
= χ2
(n − 1) and
(m − 1)S2
Y
σ2
Y
= χ2
(m − 1) (4.7.1093)
As the Xi are independent of the Yj (assumption!), the variables:
(n − 1)S2
X
σ2
X
and
(m − 1)S2
Y
σ2
Y
(4.7.1094)
are independent from each other. We can therefore apply the Fisher-Snedecor distribution:
U =
(n − 1)S2
X
σ2
X
; k = n − 1 and V =
(m − 1)S2
Y
σ2
Y
; l = m − 1 (4.7.1095)
Therefore we get:
U/k
V/l
=
(n−1)S2
X
σ2
X
/(n − 1)
(m−1)S2
Y
σ2
Y
/(m − 1)
=
S2
X
σ2
X
S2
Y
σ2
Y
=
S2
Xσ2
Y
S2
Y σ2
X
= Fn−1,m−1(x) (4.7.1096)
Finally:
σ2
X
σ2
Y
=
1
Fn−1,m−1(x)
S2
X
S2
Y
(4.7.1097)
This theorem allows us to deduce the confidence interval of the ratio of two variances when
the sample mean is known. Since Fisher function is not symmetrical, the only possibility to
make the inference is to use numerical calculation and then we will note for a given confidence
interval the Fisher test as follows:
1
Fα/2(n − 1, m − 1)
S2
X
S2
Y
σ2
X
σ2
Y
1
F1−α/2(n − 1, m − 1)
S2
X
S2
Y
(4.7.1098)
keeping in mind that its use implicitly requires constraints of Normality of the studied variables.
R.A. Fisher (1890-1962) is, as Karl Pearson, one of the main founders of the modern theory of
statistics. Fisher studied at Cambridge, where he obtained in 1912 a degree in astronomy. It
is by studying the theory of error in astronomical observations that Fisher became interested in
statistics. Fisher is the inventor of the branch of statistics named analysis of variance.
In the early 20th century, R.A. Fischer therefore develops the very important methodology of
experimental design (see section Industrial Engineering). To confirm the usefulness of a factor,
he developed a test to ensure that the different samples are of different natures. This test is
based on the analysis of variance (of the samples), and named also ANOVA but for normalized
analysis of variance.

Draft
Let us take k samples of n random values each. Each of the values being considered as an
observation or measurement of something or on the basis of something (a different place or a
different object... in short: a single factor of variability between samples!). We will have a total
of N observations (measurements) given by:
N = kn (4.7.1099)
If each sample has an equal number of values n (sample size) such that n1 = n2 = ... = nk we
then speak of balanced plan or balanced design with k levels (or k modalities).
Remark
If we have more factors of variability (e.g. each place compares himself to different
laboratories), then we will talk about multifactorial ANOVA. Therefore, if there are
only two sources of variability, we talking about two factor ANOVA (see below for
details on the various of two-factor ANOVA).
We will consider that each of the k samples is sampled (follows) a random variable following
Normally distributed:
Factor 1
Sample 1 Sample 2 Sample ...i Sample k
x11 x12 . . . x1k
x21 x22 . . . x2k
. . . . . . xij . . .
xn1 xn2 . . . xnk
Average: ¯x1 Average: ¯x2 Average: ¯xi Average: ¯xk
Table 4.29 – Typical cross table of a one factor ANOVA
In terms of testing, we test whether the mean of the k samples of size n are equal under the as-
sumption that their variances are equal. What we write as under hypothesis notation as follows:
H0 : µ1 = µ2 = . . . = µk = µ
H1 : ∃i, j µi = µj
(4.7.1100)
In other words, the samples are representative of the same population (i.e. of a same statistical
law). That is to say, the variations between the values of different samples are essentially due
to the hazard. For this we study the variability of results in the samples and between samples.
It is exactly the same as asking (formulation found in some publications or books):
H0 : σµ = 0
H1 : σµ 0
(4.7.1101)
So we will for the rest denote by i the index number of the sample (1 to k) and j the index of
the observation (from 1 to n). Therefore xij will be the value of the j-th observation of the data
sample number i (we chose to reverse the usual notation so be careful not to mislead afterwards
... we’re sorry ... it was bad choice me made when we started to write this book!).

Draft
According to the above hypothesis, we have:
xij = N(µi, σ2
) (4.7.1102)
We will denote by ¯xi the empirical/estimated (arithmetic) average of the sample i (often called
marginal average):
¯xi =
1
ni
ni
j=1
xij ⇒
ni
j=1
xij = ni ¯xi (4.7.1103)
and ¯¯x the empirical/estimated (arithmetic) average of the N value (therefore the average of the
¯xi) and then given by:
¯¯x =
1
N
k
i=1
ni
j=1
xij =
1
N
k
i=1
(ni ¯xi) (4.7.1104)
Using the properties of the mean and variance already proved above we know that:
¯xi = N µi,
σ2
i
ni
and ¯¯x = N µ,
σ2
n
(4.7.1105)
with µ which is the average of the true averages (expected means) µi:
µ =
1
N
k
i=1
µi (4.7.1106)
Let us now introduce three important variances:
1. The Total variance as being intuitively the estimated unbiased variance unbiased con-
sidering the set of N observations as a single sample:
VT =
ij
(xij − ¯¯x)2
N − 1
(4.7.1107)
where the numerator is named sum of the squares of total differences.
2. The variance between samples (that is to say between the averages of the samples) is
also intuitively the variance estimator of the averages of samples:
VE = i
(¯xi − ¯¯x)2
k − 1
(4.7.1108)
where the numerator term is named sum of the squared differences between samples.
As we have proved that if all variables are identically distributed (same variance and same
mean) and independent the variance of individuals is n times that of the average:
σ ¯X =
σ
√
n
⇔ V( ¯X)n = V(X) (4.7.1109)

Draft
then the variance of observations (random variables in a sample) is given by:
VA = nVE =
n
i
(¯xi − ¯¯x)2
k − 1
(4.7.1110)
We then have therefore above the hypothesis of equality of variances that is expressed in
mathematical form for the developments that will follow.
3. The residual variance is the effect of the uncontrolled factors. It is by deﬁnition the
average of the sample variances (its like: standard error):
VR =
1
k i
j
(xij − ¯xi)2
n − 1
=
ij
(xij − ¯xi)2
k(n − 1)
(4.7.1111)
where the numerator term is named sum of squared residuals errors or even more often
residual error.
Finally, these indicators are sometimes summarized as follows:
QT =
ij
(xij − ¯¯x)2
ddlT = N − 1 = kn − 1 VT =
QT
ddlT
QA = n
i
(¯xi − ¯¯x)2
ddlA = k − 1 VA =
QA
ddlA
QR =
ij
(xij − ¯xi)2
ddlR = k(n − 1) VR =
QR
ddlR
(4.7.1112)
Note that if the samples do not have the same size (which is rare in practice), then we have:
QT =
ij
(xij − ¯¯x)2
ddlT = N − 1 VT =
QT
ddlT
QA = n
i
(¯xi − ¯¯x)2
ddlA = k − 1 VA =
QA
ddlA
QR =
ij
(xij − ¯xi)2
ddlR =
i
(ni − 1) VR =
QR
ddlR
(4.7.1113)
Remarks
R1. The term QT is often indicated in the industry by the acronym SST meaning in
Total Sum of Squares or more rarely TSS for Total Sum of Squares.
R2. The term QA is often indicated in the industry by the acronym SSB meaning in
Sum of Squares Between (samples) or more rarely SSk for Sum of Squares Between
treatments.
R3. The term QR is often indicated in the industry by the acronym SSW meaning in Sum
of Squares Within (samples) or more rarely SSE for Sum of Squares due to Errors.

Draft
Let us indicate that we often see in the literature (we will use a little further below this notation):
QR =
ij
(xij − ¯xi)2
=
ni − 1
ni − 1 ij
(xij − ¯xi)2
=
k
i=1
ni
j=1
(xij − ¯xi)2
=
k
i=1
(ni − 1)
ni
j=1
(xij − ¯xi)2
ni − 1
=
k
i=1
(ni − 1)S2
i
(4.7.1114)
so with the unbiased estimator of the variance of the observations:
S2
i =
1
ni − 1
ni
j
(xij − ¯xi)2
(4.7.1115)
Before going further, let us stop on the residual variance. We then have for samples that are not
of the same size:
VR =
ij
(xij − ¯xi)2
i
(ni − 1)
(4.7.1116)
This writing is often named pooled variance.
Let us now open a small important parenthesis... Let’s take the case of two samples only:
VR =
j
(x1j − ¯x1)2
+
j
(x2j − ¯x2)2
(n1 − 1) + (n2 − 1)
=
j
(x1j − ¯x1)2
+
j
(x2j − ¯x2)2
n1 + n2 − 2
(4.7.1117)
Therefore by introducing the maximum likelihood estimator of the variance:
VR =
(n1 − 1)S2
1 + (n2 − 1)S2
2
n1 + n2 − 2
(4.7.1118)
We can also observe that in the speciﬁc case where n1 = n2 = n:
VR =
nS2
1 − S2
1 + nS2
2 − S2
2
2n − 2
=
n(S2
1 + S2
2) − (S2
1 + S2
2)
2(n − 1)
=
(n − 1)(S2
1 + S2
2)
2(n − 1)
=
S2
1 + S2
2
2
(4.7.1119)
So:
S2
1 VR S2
2 (4.7.1120)
Now suppose we want to compare with a conﬁdence interval the mean of two populations with
different variance to know whether they are different or not.
We currently know two tests to check the averages. The Z-test and T-test. As in the industry
(practice) it is rare that we have time to take large samples, let’s focus on the second we had
proved above:
¯X −
S
√
n
Tα/2(n − 1) ≤ µ ≤ ¯X +
S
√
n
Tα/2(n − 1) (4.7.1121)

Draft
And also recall that:
S ¯X =
S
√
n
(4.7.1122)
Now let us recall that we have proved that if we have two random variables of distribution:
N(µ1, σ2
1) N(µ2, σ2
2) (4.7.1123)
then the subtraction of the averages gives (property of stability of the average):
N µ1 − µ2, σ2
1 + σ2
2 (4.7.1124)
So for the difference average of two random variables coming from two population samples we
obtain directly:
N ¯X2 − ¯X1, S2
¯X1
+ S2
¯X2
= N

 ¯X2 − ¯X1,
S2
1
n1
+
S2
2
n2

 (4.7.1125)
And now the idea is to take the approximation (under the assumption that the variances are
equal):
N

 ¯X2 − ¯X1,
S2
1
n1
+
S2
2
n2

 ∼= N ¯X2 − ¯X1,
VR
n1
+
VR
n2
= N ¯X2 − ¯X1, VR
1
n1
+
1
n2
(4.7.1126)
This approximation is named homoscedastic hypothesis.
We then have the following conﬁdence interval (assuming that we know only an estimation of
the variance) remembering that the subtraction or the sum of two independent random variables
implies that their variances always add (so it is the same for degrees of freedom of the Student
law as we have proved above due to the direct connection with the law of the Chi-2):
¯X2 − ¯X1 − VR
1
n1
+
1
n2
Tα/2(n − 1) µ2 − µ1
¯X2 − ¯X1 + VR
1
n1
+
1
n2
Tα/2(n − 1)
(4.7.1127)
with:
n = n1 + n2 (4.7.1128)
As the idea in practice is often to test the equality of expected means (and therefore that their
difference is zero) from the known estimators then:
¯X2 − ¯X1 − VR
1
n1
+
1
n2
Tα/2(n − 2) 0 ¯X2 − ¯X1 + VR
1
n1
+
1
n2
Tα/2(n − 2)
(4.7.1129)

Draft
In most software on the market, the result is given only from the fact that quantile T that we get
is included in the Tα/2 corresponding to the given conﬁdence interval given by (for reminder):
Tcalc
(n − 2) =
¯X2 − ¯X1 − (µ2 − µ1)
VR
1
n1
+ 1
n2
(4.7.1130)
in the case of the homoscedastic hypothesis (equality of variances homogeneity of variances).
Remarks
The latter relation is named independent two-sample T-test, or homoscedastic T-test
or T-test of equality of 2 observations expectations with equal variances or more sim-
ply but somewhat abusively 2-sample T-Test with samples of different sizes and equal
variances. Often in the literature, both theoretical means are equal when comparing. It
follows that we then have:
Tcalc
(n − 2) =
¯X2 − ¯X1
VR
1
n1
+ 1
n2
=
¯X2 − ¯X1
SR
1
n1
+ 1
n2
(4.7.1131)
Otherwise, in the more general case of the assumption of heteroscedasticity (not equality of
variances), we explicitly write (we’ll come back on this later during our study of the Welch
test...):
Tcalc
(n − 2) =
¯X2 − ¯X1 − (µ2 − µ1)
S2
1
n1
+
S2
2
n2
(4.7.1132)
Therefore:
|Tcalc
(n − 1)| |Tα/2(n − 1)| (4.7.1133)
Remarks
The ante-previous relation is named independent two-sample T-test, or heteroskedas-
tic T-test or even test of equality of expected means: two observations with different
variances. If the sizes of the samples are equal and that the variances are also equal and
that we assume the two expected mean equal during the comparison, it follows that we
then have:
Tcalc
(n − 2) =
¯X2 − ¯X1
S1,2
2
n
(4.7.1134)
This ﬁnish, we close this parenthesis and return to our main topic... We were therefore at the

Draft
following table if you remember:
QT =
ij
(xij − ¯¯x)2
ddlT = N − 1 = kn − 1 VT =
QT
ddlT
QA = n
i
(¯xi − ¯¯x)2
ddlA = k − 1 VA =
QA
ddlA
QR =
ij
(xij − ¯xi)2
ddlR = k(n − 1) VR =
QR
ddlR
(4.7.1135)
Where we have in the case of samples of the same size:
ddlT = ddlA + ddlR = k − 1 + k(n − 1) = k − 1 + kn − k = kn − 1 (4.7.1136)
and also the total error that is the sum of the errors of the averages (between classes) and of the
residual error (intra-class) and this that the samples are of the same size or not:
QT = QA + QR (4.7.1137)
Indeed:
QT =
ij
(xij − ¯¯x)2
=
i j
((xij − ¯xi) + (¯xi − ¯¯x))2
i j
(xij − ¯xi)2
+
i j
(¯xi − ¯¯x)2
+ 2
i j
(xij − ¯xi) (¯xi − ¯¯x)
(4.7.1138)
But we have:
i j
(xij − ¯xi) (¯xi − ¯¯x) =
i
(¯xi − ¯¯x)
j
(xij − ¯xi) = 0 (4.7.1139)
Because:
j
(xij − ¯xi) = 0 ∀i (4.7.1140)
Therefore
QT =
i j
(xij − ¯xi)2
+
i j
(¯xi − ¯¯x)2
= QR + QA (4.7.1141)
Now, under the strong assumption (which we will be absolutly required a little further below)
that the true variances are linked such as:
σ2
R = σ2
A = σ2
T = σ2
(4.7.1142)
and therefore that respective estimators are asymptotically equal ... which in practice is ap-
proximately true only when certain conditions are met (which is why it is absolutely necessary
before making an ANOVA to run an a priori calculation of the power and the sample size of the
ANOVA test) we have:
QR
σ2
=
ij
(xij − ¯xi)2
σ2
= χ2
N−k (4.7.1143)

Draft
that follows immediately from the proof that we made during our study of statistical inference
with the Chi-square law where we got (for refresh!):
χ2
n−1 =
(n − 1)S2
σ2
=
(n − 1)
1
n − 1
n
i=1
(Xi − ¯X)2
σ2
=
n
i=1
(Xi − ¯X)2
σ2
(4.7.1144)
To determine the number of degrees of freedom of the Chi-square law of:
QA
σ2
= i
ni (¯xi − ¯¯x)2
σ2
(4.7.1145)
We will use the fact that (by the same reasoning as for the ante-previous relation):
QT
σ2
=
ij
(xij − ¯¯x)2
N − 1
= χ2
N−1 (4.7.1146)
and since that QT = QA + QR, then we must have:
QT
σ2
=
QA + QR
σ2
=
QA
σ2
+
QR
σ2
= χ2
? + χ2
N−k = χ2
N−1 (4.7.1147)
It follows by the linearity property of the Chi-Square law:
QA
σ2
= i
ni (¯xi − ¯¯x)2
σ2
= χ2
k−1 (4.7.1148)
So to summarize we have:
QA
σ2
= χ2
k−1 and
QR
σ2
= χ2
N−k (4.7.1149)
Now comes the Fisher law in the assumption that the variances are equal (and measurement
Normally distributed)! Because:
(k − 1)S2
A
σ2
= χ2
k−1 and
(N − k)S2
R
σ2
= χ2
N−k (4.7.1150)
What we want to do is to see if there is a difference between the variance of the averages
(between classes) and the residual variance (intra-class). To compare two variances when the
true averages are unknown we saw that the best was to use the Fisher test. But, we proved in
our study of the Fisher law a little above that:
σ2
X
σ2
Y
=
1
Fn−1,m−1(x)
S2
X
S2
Y
(4.7.1151)
where in our case study:
σ2
A
σ2
R
= 1 ⇒ Fk−1,N−k =
S2
A
S2
R
=
VA
VR
=
i
ni(¯xi − ¯¯x)2
k − 1
ij
(xij − ¯xi)2
N − k
=
QA
k − 1
QR
N − k
(4.7.1152)

Draft
Since there are dozens of different types of ANOVA we have to understand very good this choice
of the simplest ANOVA we are studying right now! Thus, if the averages are the same, the null
hypothesis is then that variance ratio is equal to unity (under the above assumptions already
mentioned far above). If F is too large at a given threshold, then we reject the null hypothesis of
equality of means (because verbatim variances will be strongly different). So it seems here logic
to compare the variances between groups (numerator) with that in the groups (denominator) but
as we shall see this is not always the choice to be made (especially in hierarchical ANOVA)!
Given the assumption of the ﬁrst equality of the above relation (the one that precedes the impli-
cation), we understand at the same time much better the great sensitivity of the ANOVA results
to the non equality of true variances!
Let us also indicate that the previous relation:
Fk−1,N−k =
QA
k − 1
QR
N − k
(4.7.1153)
is often given in the literature as follows:
Fk−1,N−k =
MSk
MSE
(4.7.1154)
where MSk is named Mean Square for treatments and MSE Mean Square for Error. This
ratio will therefore give us the value of the random variable F (whose support is for reminder
bounded at zero). Regarding the choice of the test (right/left one-sided or bilateral), note that if
the averages are really equal, then for all i:
(xi − ¯¯x) = 0 (4.7.1155)
So in this case:
Fk−1,N−k =
i
ni(¯xi − ¯¯x)2
k − 1
ij
(xij − ¯xi)2
N − k
=
i
ni · 0
k − 1
ij
(xij − ¯xi)2
N − k
= 0 (4.7.1156)
Which brings us of course to immediately adopt a right-sided test!
Otherwise, in general, the interpretation of this fraction is basically as following: This is the
ratio (normalized to the number of degrees of freedom) of the sum of the error of the averages
(between classes) and that of the residual error (intra-class) or in other words the ratio of the
interclass variance by the residual variance. The ratio thus follows a Fisher distribution with
two parameters given by the degrees of freedom of the respective classes.
Remark
If there are only two populations (samples), we must understand that then that use of the
Student T-test is more than enough and is perfectly equivalent! In fact, the ANOVA is an
indirect comparison of means, the Student T-test a direct comparison... so it is obvious
to guess which one is better in this particular situation!

Draft
All calculations we have made until now are very often represented in softwares in a standard
table form as represented below (this is for example how Microsoft Excel 11.8346 or Minitab
15.1.1 give the results):
Source Sum of squares χ2 df Average of squares F Critical F
Inter-Class QA =
i
ni (¯xi − ¯¯x)2
k − 1 MSk =
QA
k − 1
MSk
MSE P(F Fk−1,N−k)
Intra-Class QR =
ij
(xij − ¯xi)2
N − k MSE =
QR
k − 1
Total QT =
ij
(xij − ¯¯x)2
N − 1
Table 4.30 – One way fixed factor ANOVA table
The null hypothesis will not be rejected if the value of:
Fk−1,N−k =
MSk
MSE
(4.7.1157)
is smaller or equal to the quantile of the distribution F that corresponds to the cumulative
probability to 1 subtracted from confidence level α and denoted by Fc.
Remarks
R1. Note that in practice, inter-class variance is often named inter-laboratory variance
and the intra-class variance is verbatim often named intra-laboratory variance.
R2. There are, in this early 21st century more than 50 existing tests or procedures
comparison for the variance. The opinion varies among practitioners relatively to their
relevance and the effectiveness of homogeneity of variance tests (HVT). Some argue
that they are absolutely necessary to be applied before doing any ANOVA, others say
that these tests are anyway of poor performance, the ANOVA being anyway more robust
to homoscedasticity than the differences than can be detected by HVT, particularly
in case of non-Normality. In fact, all these issues are related to the Behrens-Fisher
problem, which is that the comparison of means without assuming the equivariance.
However among the fifty existing tests, several comparative studies have highlighted the
following efficient tests that we will present further below: Bartlett’s test, Levene and
Brown-Forsythe.
R3. When certain levels of a factor are combined into one to be compared to a reference
level statisticians then talk about creating contrasts. For example a level: control
group is compared to a level that is the union of several levels that are initially test
group 1, test group 2 and test group 3. In the latter case we are dealing obviously a
non-equilibrated ANOVA.

Draft
4.7.13.6 Analysis of Variance with two fixed factors without repetitions
We will now see the concept of interaction that is essential to understand what is behind the fixed
two-factor ANOVA (or ANOVA with two fixed categorical variables) without and especially
later with repetitions. Indeed, it is only with two-factor ANOVA with repetitions - by mathe-
matical construction - that can be statistically (under given assumptions) consider objectively if
two or more factors interact significantly together.
We must therefore, before moving to the pure mathematical part, introduce some concepts!
Definitions
D1. We say that there is no interaction when the average of responses by a factor based on its
level varies by the same amount and with the same sign as the average of the responses of
another factor depending on its levels. Then we say that: the response interaction curves
in the diagram are parallel.
Remark
The parallelism of response curves is normal in the situation of no interaction be-
cause it means that whatever the level of one or the other factors, the variation (if
there is any) of the response will always be the same amplitude. What is character-
istic of independence (at least locally).
D2. We say that two factors are in interaction when the average responses by a factor de-
pending on its levels do not change by the same magnitude and / or not with the same
sign as the average of the responses of another factor according to their levels. Then we
say that: the response curves in the interactions diagram are not parallel.
Remark
The absence of interaction is a very strong assumption and a rarely observed. Often
we have interactions or strong interactions.
To understand the concept, we will use small examples have no repeated measures to such we
will have a simple qualitative idea of the phenomenon, but by no means a scientific approach of
the concept of interaction (that will be study later in details).
For every small example below we visualize the situations by means of two types of represen-
tations: a graph illustrating the main effects on the one hand and a pattern of the interactions on
the other hand.

Draft
Consider the following small table with two factors at two levels (explanatory variables) in-
cluding four cells (variable of interest):
Factor 2
Factor 1 Level 1 Level 2
Level 1 3 3
Level 2 3 3
Table 4.31 – First example of 2-factor ANOVA without repetition
We will get as result with software such as Minitab:
Figure 4.119 – Main effects plot without interactions in Minitab 15
We see well that no factor has a major effect on anything. What is relatively obvious given the
content of the previous table...
The diagram of interactions (often named proﬁler in the industry) gives:

Draft
Figure 4.120 – Interactions plot without interactions... in Minitab 15
where we can see that the factors do not interact between them (or neutralize themselves it
depends...). Then we say that there is (a priori) no effect or no interaction (locally). In fact
in some experiments the absence of interaction is a very strong assumption and therefore very
reare. This is why we must pay attention to the chosen words when interpreting interaction
plots (because do not to go through pure calculations is dangerous for this step or simply not
scientiﬁc!).
Now consider the following table:
Factor 2
Level 1 2 2
Level 2 4 4
Table 4.32 – Second example of 2-factor ANOVA without repetition
It seems obvious that the factor one through considered with its levels seems to have an inﬂuence
on the answer. But let us see the different representations:

Draft
Figure 4.121 – Main effects and interactions plot with interactions in Minitab 15
Let us comment the first graph as proposed by a reader:
This includes two parts: the left one analyzes the effects of Factor 1 through its two levels; that
of the right do the same for the Factor 2.
Have a closer look to the left part:
We see two points connected by a line segment. Here the first point, the one for the level 11,
is located in the ordinate 2 while the second point, that for level 2, is located in the ordinate
4. Now remember that each point represents an average. And the ordinate of the first point is

Draft
located at the average of (2 + 2)/2 = 2.
Having said that and hoping this helped to a better understanding, let us continue...
It is quite clear in the chart above that only the level of Factor 1 influences the answer, while
Factor 2 has no effect on the answer. Then we say that: there is a main effect (locally) of the
Factor 1.
On the diagram of interactions, we have the same information but in a different form. We see
that regardless of the level of Factor 2, the answers are horizontal and thus that it does not
influence the results. We are then in a situation where (a priori) the main effect is (locally)
Factor 1 and there is an absence of interactions between factors. Now consider the following
table:
Factor 2
Level 1 4 2
Level 2 4 2
Table 4.33 – Third example of 2-factor ANOVA without repetition
This time we can observe that the Factor 2 has an influence but not the Factor 1. But let us also
see this with our two types of representations:

Draft
We see well on the top diagram that the Factor 1 has no influence. On the bottom diagram it is
less obvious but the superposition of the two lines indicates that the Factor 1 has no influence.
Then we say that there is: (a priori) main effect (locally) of Factor 2 and absence of interactions
between factors.
Factor 2
Level 1 3 1
Level 2 5 3
Table 4.34 – Fourth example of 2-factor ANOVA without repetition
We see that both factors have an influence on the answer. We can clearly see it on the both
diagrams below:

Draft
We can see very well on the top diagram that the Factor 1 has an inﬂuence on the answer and
that we have the same conclusion for the Factor 2 (and furthermore to in the same magnitude
regardless of the direction!). On the bottom diagram it is less obvious, but the same conclu-
sion is valid. We then say that: (a priori) both factors are (locally) signiﬁcant and without
interactions.

Draft
Factor 2
Level 1 2 4
Level 2 4 2
Table 4.35 – Fifth example of 2-factor ANOVA without repetition
which in this form is not always obvious to interpret. But with the diagrams we immediately
have a more relevant information:

Draft
We see well on the top diagram that none of the factors has an effect in average on the answer a
priori (same diagram as at the very beginning of this series of examples with the same mean!).
The bottom diagram gives us additional information by cons (!!!): The factors have a cross-
inﬂuence and as this cross-inﬂuence is of the same magnitude, the effects cancel out. Then we
say that: (a priori) the two factors are (locally) in interaction in F1 * F2.
Factor 2
Level 1 1 3
Level 2 5 3
Table 4.36 – Sixth example of 2-factor ANOVA without repetition
which in this form is not also always obvious to interpret. But with the diagrams we immediately
have a more relevant information:

Draft
We observe well on the top diagram that the Factor 1 appears to have an influence and the
factor 2 no (in average!). The bottom interactions diagram gives us, too, once again, additional
information (!!!): It is that the factors are interacting. We say then that we hae (a priori) two
factors (locally) in interaction F1 * F2 where the influence of Factor 1 is significant.
Factor 2
Level 1 3 3
Level 2 5 1
Table 4.37 – Seventh example of 2-factor ANOVA without repetition
We see that both factors influence the answer. We see that well on the both diagrams below:

Draft
Here we say we have: (a priori) the two factors (locally) interacting F1 * F2 where the influence
of Factor 2 is significant.
And finally a last example:

Draft
Factor 2
Level 1 1 5
Level 2 5 1
Table 4.38 – Eighth and last example of 2-factor ANOVA without repetition
Which give us the two diagrams:

Draft
Remark
A belief (commonly spread) of people who have a lack experience in laboratories is to
think that for an interaction to be significant it is necessary that the both factors that
compose it are also significant.
After all these diagrams and tables, let us turn back to the mathematical part:
We saw earlier how to perform a one-way fixed factor analysis of variance. To recap, this
consists to a test for equality of means for k independent samples of n random variables each
(in the case where all samples have the same number of measures: a balanced ANOVA). Each
sample is regarded as an experiment on a different subject then considered therefore as an
independent variable factor!!!
However it happens in the reality that we do for each sample vary a second parameter, then
considered a second variable factor but independent of the first one. We speak then of course
analysis of variance with two factors. In addition, we will consider in the first instance to
simplify the calculations that the random variables are independent! Therefore a factor has no
influence over the other has we already mention it !!! In other words there is no interaction
between factors. We speak then of a two-factor ANOVA without interactions.
To determine the formulation of the test to be performed, remember that for the analysis of
variance with factor, we decomposed the total variance in the sum of the variance of average
(inter-class) and of the residual variance (intraclass) such that:
QT = QA + QR (4.7.1158)
explicitly we comparee the samples i = 1...k:
QT =
ij
(xij − ¯¯x)2
=
i j
(xij − ¯¯x)2
=
i j
((xij − ¯xi) + (¯xi − ¯¯x))2
=
i j
(xij − ¯xi)2
+
i j
(¯xi − ¯¯x)2
+ 2
i j
(xij − ¯xi)(¯xi − ¯¯x)
(4.7.1159)
which had given us in the end:
QT =
i j
(xij − ¯xi)2
+
i j
(¯xi − ¯¯x)2
= QR + QA (4.7.1160)
For the ANOVA with two fixed factors we will start form the table below:
Factor A
Factor B Sample 1 Sample 2 Sample ...j Sample r
Sample 1 x11 x12 . . . x1r Average ¯x1
Sample 2 x21 x22 . . . x2r Average ¯x2
Sample i . . . . . . xij . . . Average ¯xi
Sample k xn1 xn2 . . . xkr Average ¯xk
Average: ¯x1 Average: ¯x2 Average: ¯xj Average: ¯xr ¯¯x
Table 4.39 – Typical cross structure for the analysis of variance of 2 fixed factors without repetitions

Draft
for which a laboratory, factor kept fixed while we will vary the other will be named the blocking
factor and the other will be named Treatment factor and in practice we will ensure that it
does is not always performed in the same order to eliminate potential inertia when changing
from one treatment to the other effects (some authors designate the two fixed factor ANOVA
without interactions by the terms randomized block design R.B.D.).
To continue the whole trick is to break down the total variance by comparing the average of the
rows (observations) indexed this time with i = 1...k and of columns (samples) indexed with
j = 1..r relatively to the total average such as:
QT =
ij
(xij − ¯¯x)2
=
ij
(xij − ¯¯x)2
=
i j
[(¯xi − ¯¯x) + (¯xj − ¯¯x) + (xij − ¯xi − ¯xj + ¯¯x)]2
=
ij
[((¯xi − ¯¯x) + (¯xj − ¯¯x))2
+ 2 ((¯xi − ¯¯x) + (¯xj − ¯¯x)) (xij − ¯xi − ¯xj + ¯¯x)
+ (xij − ¯xi − ¯xj + ¯¯x)2
]
=
ij
[(¯xi − ¯¯x)2
+ 2(¯xi − ¯¯x)(¯xj − ¯¯x) + (¯xj − ¯¯x)2
+ 2((¯xi − ¯¯x) + (¯xj − ¯¯x))(xij − ¯xi − ¯xj + ¯¯x)
+ (xij − ¯xi − ¯xj + ¯¯x)2
]
(4.7.1161)
But we have in a first time:
ij
2(¯xi − ¯¯x)(¯xj − ¯¯x) = 2
i
(¯xi − ¯¯x)
=0
j
(¯xj − ¯¯x)
=0
= 0 (4.7.1162)
So it remains:
QT =
ij
[(¯xi − ¯¯x)2
+ (¯xj − ¯¯x)2
+ 2((¯xi − ¯¯x) + (¯xj − ¯¯x))(xij − ¯xi − ¯xj + ¯¯x)
+ (xij − ¯xi − ¯xj + ¯¯x)2
]
(4.7.1163)
But we also have:
ij
2((¯xi − ¯¯x) + (¯xj − ¯¯x))(xij − ¯xi − ¯xj + ¯¯x)
= 2
ij
((¯xi − ¯¯x) + (¯xj − ¯¯x))(xij − ¯xi − ¯xj + ¯¯x)
= 2
ij
(¯xi − ¯¯x)(xij − ¯xi − ¯xj + ¯¯x) + 2
ij
(¯xj − ¯¯x)(xij − ¯xi − ¯xj + ¯¯x)
(4.7.1164)
For the rest, let us indicate first that relatively to our above table, we have:
¯¯x =
1
rk ij
xij
¯xi =
1
r i
xi
¯xj =
1
k j
xj
(4.7.1165)

Draft
It follows then that:
ij
(¯xi − ¯¯x)(xij − ¯xi − ¯xj + ¯¯x) =
i
(¯xi − ¯¯x)
j
xij −
j
¯xi −
j
¯xj +
j
¯¯x
=
i
(¯xi − ¯¯x) r¯xi − ¯xi
j
1 −
j
1
k i
xij + ¯¯x
j
1
=
i
(¯xi − ¯¯x) r¯xi − ¯xir −
1
k ij
xij + ¯¯xr =
i
(¯xi − ¯¯x)(−r¯¯x + ¯¯xr)
=
i
(¯xi − ¯¯x) · 0 = 0
(4.7.1166)
and then it comes immediately that we have likewise:
ij
(¯xj − ¯¯x)(xij − ¯xi − ¯xj + ¯¯x) = 0 (4.7.1167)
So it remains in the end:
QT =
ij
[(¯xi − ¯¯x)2
+ (¯xj − ¯¯x)2
+ (xij − ¯xi − ¯xj + ¯¯x)2
]
=
ij
(¯xi − ¯¯x)2
+
ij
(¯xj − ¯¯x)2
+
ij
(xij − ¯xi − ¯xj + ¯¯x)2
= k
j
(¯xk − ¯¯x)2
+ r
i
(¯xi − ¯¯x)2
+
ij
(xij − ¯xi − ¯xj + ¯¯x)2
(4.7.1168)
what we will note that on this site in the following condensed form:
QT = QA + QB + QR (4.7.1169)
where QA, QB are obviously associated with the main effects (comparison of marginal means
with the total average).
Therefore compared to the one-way fixed factor ANOVA we have an additional term for the
total variance!
In order it is clear that the first sum of the differences from the first column factor:
ij
(¯xi − ¯¯x) (4.7.1170)
will have just as for the one-way fixed factor ANOVA k − 1 degrees of freedom. That is to say,
under the same assumptions as the one-way fixed factore ANOVA:
ij(¯xi − ¯¯x)
σ2
= χ2
k−1 (4.7.1171)
The second sum of the differences from the second factor (line factore):
ij
(¯xj − ¯¯x) (4.7.1172)

Draft
is new but we prove in a perfectly identical way to the first one it will have k − 1 degrees of
freedom. That is to say, under the same assumptions as the one-way fixed factor ANOVA:
ij(¯xj − ¯¯x)
σ2
= χ2
r−1 (4.7.1173)
For the third sum that follows mandatory also a chi-square law (since the total variance follows
a chi-square law and that the first two terms of the sum also!):
ij
(xij − ¯xi − ¯xj + ¯¯x)2
(4.7.1174)
it’s a little bit trickier ... but there’s a trick in the physicist way of life...! We know from our study
of the one-way fixed factor ANOVA that the sum of the degrees of freedom of each term must
equal the total number of degrees of freedom. In other words, we must have for the two-way
fixed factor ANOVA:
ddlT = N − 1 = kr − 1 = (k − 1) + (r − 1)+??? (4.7.1175)
So obviously what is missing is equal to:
??? = kr − 1 − [(k − 1) + (r − 1)] = kr − 1 − k + 1 − r + 1 = kr − k − r + 1 = (k − 1)(r − 1)
(4.7.1176)
Therefore:
ij(xij − ¯xi − ¯xj + ¯¯x)2
σ2
= χ2
(k−1)(r−1) (4.7.1177)
We then have the following table:
QT =
ij
(xij − ¯¯x)2
ddlT = N − 1 = kr − 1 VT =
QT
ddlT
QA = k
j
(¯xj − ¯¯x)2
ddlA = r − 1 VA =
QA
ddlA
QB = r
i
(¯xi − ¯¯x)2
ddlB = k − 1 VB =
QB
ddlB
QR =
ij
(xij − ¯xi − ¯xj + ¯¯x)2
ddlR = (k − 1)(r − 1) VR =
QR
ddlR
(4.7.1178)
Finally, the rest is exactly the same as for the one-way fixed factor ANOVA just that we have
two tests to do this time that are:
Fk−1,(k−1)(r−1) =
QA
k − 1
QR
(k − 1)(r − 1)
and Fr−1,(k−1)(r−1) =
QB
r − 1
QR
(k − 1)(r − 1)
(4.7.1179)
The choice above seems intuitively practical...

Draft
All calculations we have made above are often represented in software in a standard form of
a table whose form and content is presented below (this is how Minitab 15.1.1 and Microsoft
Excel 11.8346 present it for example):
Sum of squares χ2 df Average of squares F Critical F
QA = k
j
(¯xj − ¯¯x)2
k − 1 MSkA = QA
k−1
MSkA
MSE P(F Fk−1,(k−1)(r−1))
QB = r
i
(¯xi − ¯¯x)2
r − 1 MSkB = QB
r−1
MSkB
MSE P(F Fr−1,(k−1)(r−1))
QR = ij (xij − ¯xi − ¯xj + ¯¯x)2
(k − 1)(r − 1) MSE = QR
(k−1)(n−1)
QT =
ij
(xij − ¯¯x)2
N − 1
Table 4.40 – Two way fixed factor ANOVA table without repetitions
and the condition of acceptance of the null hypothesis of equality of means for each factor is the
same as for the one-way fixed factor ANOVA (see the server of exercises for a detailed practical
example with Microsoft Excel 11.8346).
So we have two Fisher tests to know whether if factor A (respectively B) have a significant
influence on the measures or not.
Obviously, in the above developments, factors A and B are interchangeable in the developments
by symmetry!
4.7.13.7 Analysis of Variance with two fixed factors with repetitions
So far we have examined ANOVA on experiments with one or two fixed factors : one or two
categorical variables not sampled from a populations). In the case of two factors, we considered
that for each combination of factors we only had one measure (cell). But it can happen (and this
is better!) that have several measures for one combination of factors!
We name this type of study of experimental design with repeated measures and the results
will be processed with an analysis of variance for repeated measures with two fixed factors
and interactions! This is an extremely important tool as it allows validation studies by several
independant laboratories (or employees) and is also associated with many other statistical tools
such as the study of reproducibility and repeatability (RR Study) to name only the best known
cases in the industrial field.
You must understand that it is mandatory in the field of statistics to involve interactions between
factors systematically when we are dealing with an experience having repeated measures. This
for the simple reason that the mathematical interaction term only appears in this situation!
Thus, it may be intuitive (even before the proof) that a two fixed factor ANOVA with repeated
measures (some authors designate the two fixed factor ANOVA with interactions under the
terms generalized randomized block design GRBD) contains one double interaction, and two
main effects. A three fixed factor ANOVA with repeated measures will have one triple interac-
tion, three double interactions and 3 main effects. And so on...

Draft
Before we start, we will consider the measurement table on the next page:
Factor A
Factor B Sample 1 Sample 2 Sample ...j Sample r AVERAGE
Sample 1 x111 x112 . . . x11r
Replication 2 x211 x212 . . . x21r
Replication m . . . . . . . . . . . .
Replication n xn11 xn12 . . . xn1r
Average Sample 1 ¯x11 ¯x12 . . . ¯x13 ¯x1.
Sample 2 x121 x122 . . . x12r
Replication m . . . . . . . . . . . .
Replication n xn21 xn22 . . . xn2r
Average Sample 2 ¯x21 ¯x22 . . . ¯x2r ¯x2.
Sample i . . . . . . x1ij . . .
Replication 2 . . . . . . . . . . . .
Replication m . . . . . . . . . . . .
Replication n . . . . . . xnij xnir
Average Sample i . . . . . . ¯xij . . . ¯xi.
Sample k x2k1 x2k2 . . . x2kr
Replication m . . . . . . . . . . . .
Replication n xnk1 xnk2 . . . xnkr
Average Sample k ¯xk1 ¯xk2 . . . ¯xkr ¯xk.
AVERAGE ¯x.1 ¯x.2 ¯x.j ¯x.r ¯¯x..
Table 4.41 – Typical cross structure for the analysis of variance of 2 ﬁxed factors with repetitions
with the usual properties of the mean (for reminder):
¯xi. =
1
r
r
j=1
¯xij =
1
rn
n
m=1
r
j=1
xmij
¯x.j =
1
k
k
i=1
¯xij =
1
kn
n
m=1
k
k=1
xmij
¯x.. =
1
r
r
j=1
¯x.j =
1
k
k
i=1
¯xij =
1
krn
n
m=1 k=1
r
j=1
xmij
(4.7.1180)
And remember that that the two ﬁxed factor ANOVA without replications (and therefore without
possibility to analyze mathematically the interactions), all the trick was to to break down the
total variance by comparing the average of the lines indexed with i = 1...k and of columns
indexed with j = 1...r compared with the total average.
The idea will now be just about the same except that we will compare the mean of the lines
indexed with i = 1...k and columns indexed with j = 1...r not only in comparison to the total
average but also that of each row and of each column.

Draft
For this purpose with start from we got for the two ﬁxed factor ANOVA without replication:
QT =
ij
(xij − ¯¯x)2
=
ij
(xi − ¯¯x)2
+
ij
(xj − ¯¯x)2
+
ij
(xij − ¯xi − ¯xj + ¯¯x)2
(4.7.1181)
but whose notation will just be adapted to the context:
QT =
ij
(xij − ¯¯x..)2
= r
i
(xi. − ¯¯x..)2
+ k
j
(x.j − ¯¯x..)2
+
ij
(xij − ¯xi. − ¯x.j + ¯¯x..)2
(4.7.1182)
Obviously, with this notation the two ﬁxed factor ANOVA without replication become:
QT =
ij
(xij − ¯¯x)2
→ QT =
ij
(xij − ¯¯x..)2
QA = r
i
(xi − ¯¯x)2
→ QA = r
i
(xi. − ¯¯x..)2
QB = k
i
(xj − ¯¯x)2
→ QB = k
j
(x.j − ¯¯x..)2
(4.7.1183)
But in the present case, we must add a term for the replication and adapt the notation for the
measurements. So, without repeating all the developments (it is a bit cheeky but...) we get
directly:
QT =
mij
(xmij − ¯¯x..)2
= nr
i
(xi. − ¯¯x..)2
+ nk
j
(x.j − ¯¯x..)2
+
ijm
(xmij − ¯xi. − ¯x.j + ¯¯x..)2
(4.7.1184)
where in order, m is the replicaton of the sample i of the factor A and of sample j of factor B.
It comes then obviously the interclass variances for factors A and B that are immediate:
QA = nr
i
(xi. − ¯¯x..)2
QB = nk
j
(x.j − ¯¯x..)2
(4.7.1185)
where QA, QB are obviously again associated with main effects (comparisons of marginal av-
erages with the total average).
Now let’s play a little by introducing in the sum, in positive and negative in the last term:
ijm
(xmij − ¯xi. − ¯x.j + ¯¯x..)2
(4.7.1186)
the average of replications:
¯xij (4.7.1187)

Draft
that we will find in fine in the total sum of squares:
mij
(xmij − ¯xi. − ¯x.j + ¯¯x..)2
=
mij
(xmij − ¯xi. − ¯x.j + ¯¯x.. + ¯xij − ¯xij)2
=
mij
((¯xij − xi. − x.j + ¯¯x) + (xmij − ¯xij))2
=
mij
(¯xij − xi. − x.j + ¯¯x)2
+ 2
mij
(¯xij − xi. − x.j + ¯¯x) (xmij − ¯xij) +
mij
(xmij − ¯xij)2
= n
ij
(¯xij − xi. − x.j + ¯¯x)2
+ 2
mij
(¯xij − xi. − x.j + ¯¯x) (xmij − ¯xij) +
mij
(xmij − ¯xij)2
(4.7.1188)
Of course, we recognize rather quickly the intra-classes variance (often also named residual
error or simply in the particular case of the two fixed factor ANOVA with repetitions repeata-
bility error):
QR =
mij
(xmij − ¯xij)2
(4.7.1189)
and the term we can interpret (by comparison with the two factor ANOVA without repetitions)
as the variance of interaction:
QA×B = QA,B = n
ij
(¯xij − xi. − x.j + ¯¯x)2
= n
ij
((¯xij − ¯¯x) (4.7.1190)
If the ANOVA is balanced the term:
2
mij
(¯xij − xi. − x.j + ¯¯x..) (xmij − ¯xij) (4.7.1191)
must vanish. Let us check this:
mij
(¯xij − xi. − x.j + ¯¯x) (xmij − ¯xij) =
mij
(xmij − ¯xij) (¯xij − xi. − x.j + ¯¯x)
=
i j m
(xmij − ¯xij) (¯xij − xi. − x.j + ¯¯x)
(4.7.1192)
and therefore for i and j fixed it comes:
(¯xij − xi. − x.j + ¯¯x)
m
(xmij − ¯xij)
=0
= 0 (4.7.1193)
And therefore the summation over all i and j will be zero also by extension. Those who have a
doubt about the cancellation of the two terms of the development above, may be able to reassure
themselves by making a numerical application (enjoy!...).
Then finally:
QT =
mij
(xmij − ¯¯x..)2
= nk
j
(x.j − ¯¯x..)2
QA
+ nr
i
(xi. − ¯¯x..)2
QB
+ n
ij
(xmij − ¯xi. − ¯x.j + ¯¯x..)2
QA×B
+
mij
(xmij − ¯¯xij)2
QR
(4.7.1194)

Draft
where for reminder, n is therefore the number of replications, r the number of samples of factor
A and k and the number of samples of factor B (the latter two parameters are often mixed by
those who make calculations by hand) . Result is sometimes noted as following in the literature:
QT =
mij
(xmij − ¯x...)2
= nk
j
(x.j − ¯x...)2
+ nr
i
(xi. − ¯x...)2
+ n
ij
(xmij − ¯xi. − ¯x.j + ¯x...)2
+
mij
(xmij − ¯¯xij)2
(4.7.1195)
So in comparison to the two fixed factor ANOVA without replication, we have an additional
term for the total variance.
In the order it is clear that the first sum of the differences from the first factor column:
j
(x.j − ¯¯x..)2
(4.7.1196)
will have just likefor one way fixed factof ANOVA and two way fixed factor ANOVA without
repetition k − 1 degrees of freedom. That is to say, under the same assumptions of these latter
ANOVA, we have:
j
(x.j − ¯¯x..)2
σ2
= χ2
k−1 (4.7.1197)
The second sum of the differences relatively to the second factor line:
i
(xi. − ¯¯x..)2
(4.7.1198)
will have under the same assumptions the property:
i
(xi. − ¯¯x..)2
σ2
= χ2
r−1 (4.7.1199)
Thanks to the reasoning performed with two fixed actor ANOVA without repetition, we know
that for the interaction term:
ij
(xmij − ¯xi. − ¯x.j + ¯¯x..)2
(4.7.1200)
We have:
ij
(xmij − ¯xi. − ¯x.j + ¯¯x..)2
σ2
= χ2
(k−1)(r−1) (4.7.1201)
It remains to determine the number of degrees of freedom of the last term:
mij
(xmij − ¯¯xij)2
(4.7.1202)

Draft
To do this, we proceed in the same way as for the two fixed factor ANOVA without repetitions.
We know from our study of the one way fixed factor ANOVA that the sum of the degrees of
freedom of each term must be equal tot the total number of degrees of freedom. In other words,
we must have for the two fixed factors ANOVA:
ddlT = N − 1 = nkr − 1 = (k − 1) + (r − 1) + (k − 1)(r − 1)+??? (4.7.1203)
So what is missing is obvious:
??? = nkr − 1 − [(k − 1) + (r − 1) + (k − 1)(r − 1)] = nkr − 1 − k + 1 − r + 1 − (kr − k − r + 1)
= nkr − 1 − k + 1 − r + 1 − kr + k + r − 1 = nkr − kr = N − kr = kr(n − 1)
(4.7.1204)
Therefore:
mij
(xmij − ¯xij)2
σ2
= χ2
N−kr (4.7.1205)
We have then the following table:
QT =
mij
(xmij − ¯¯x..)2
ddlT = N − 1 = nkr − 1 VT =
QT
ddlT
QA = nk
j
(¯x.j − ¯¯x..)2
ddlA = k − 1 VA =
QA
ddlA
QB = nr
i
(¯xi. − ¯¯x..)2
ddlB = r − 1 VB =
QB
ddlB
QA×B = n
ij
(¯xij − ¯xi. − ¯x.j + ¯¯x..)2
ddlA×B = (k − 1)(r − 1) VA×B =
QA×B
ddlA×B
QR =
mij
(xmij − ¯¯x..)2
ddlR = N − kr VR =
QR
ddlR
(4.7.1206)
Finally, the rest is exactly the same as for the two-way fixed factor ANOVA without replication
with the difference that now we just have three tests this time:
Fk−1,N−kr =
QA
k − 1
QR
N − kr
Fr−1,N−kr =
QB
r − 1
QR
N − kr
F(k−1)(r−1),N−kr =
QA×B
(k − 1)(r − 1)
QR
N − kr
(4.7.1207)
Again the choice of the ratios is relatively intuitive! All calculations we have made above are
often represented in software in a standard form of a table whose form and content is presented
below (this is how Minitab 15.1.1 and Microsoft Excel 11.8346 present it for example):

Draft
Sum of squares χ2 df Average of squares F Critical F
QA = nk
j
(¯x.j − ¯¯x..)2
k − 1 MSkA = QA
k−1
MSkA
MSE P(F Fk−1,Nk−r)
QB = nr
i
(¯xi. − ¯¯x..)2
r − 1 MSkB = QB
r−1
MSkB
MSE P(F Fr−1,N−kr)
QA×B =
mij
(¯xmij − ¯xi. − ¯x.j + ¯¯x..)2
(k − 1)(r − 1) MSkAB =
QA×B
(k−1)(n−1)
MskAB
MSE
P(F F(k−1)(n−1),N−kr)
QR =
mij
(¯xmij − ¯xij)2
N − kr MSE = QR
N−kr
QT =
mij
(xmij − ¯¯x..)2
N − 1
Table 4.42 – Two way fixed factor ANOVA table with repetitions
and the null hypothesis of equality of means for each factor is the same as for one-way fixed
factor ANOVA (see the exercise server for a detailed practical example with Microsoft Excel
11.8346).
So we have three Fisher tests for to know whether for each if factor A, respectively B or inter-
action AB have a significant influence on the measures or not.
Obviously, in the above developments, factors A and B are interchangeable by symmetry!
4.7.13.8 Multifactor ANOVA with Repeated measures
The multifactor categorical variables ANOVA with repeated measures is simply the name by
which specialists refer to the following fixed-factors ANOVA:
• three factors (fixed) ANOVA with or without repetition
• four factors (fixed) ANOVA with or without repetition
• five factors (fixed) ANOVA with or without repetition
• etc.
Obviously, the ANOVA with one and two fixed factors are also part of the family of multi-
factor ANOVA. Also be aware that most statistical software manages up to ANOVA with 15
fixed factors fixed factors (categorical variables) on the assumption condition that the plan is
balanced (i.e. for each level of each factor, there is the same number of measurements). A
spreadsheet software (like Microsoft Excel for example) by default ANOVA with maximum
two fixed factors.
Okay now the reader may be disappointed (well I am also disappointed to have only one life
...) because honestly I do not want to redo all the developments seen above for the one-way
and two-way fixed factors ANOVA for 3, 4 and up to 15 factors because it would take more
than 100 A4 pages to do in a pedagogical and clear form plus it is always based on the same
mechanical development. Sadly the generalized theory of ANOVA well being much shorter, it
is indigestible for my taste.

Draft
4.7.13.9 Greaco Latin Square ANOVA
A Graeco-Latin square ANOVA is an extension of a Latin square used when the number of
block is greater than or equal to 2. Concretely, a Graeco-Latin square ANOVA can control up
to three nuisance factors and one treatment factor.
To figure out a Graeco-Latin square, we consider initially a Latin square K ⊗ K on which we
superimposed another Latin square K ⊗ K which this time is denoted by Greek letters. After-
wards in a second time during the superposition, we must ensure that the following property is
respected, each row and each column can only contain couple of letters (Latin, Greek) that are
distincts. Thus, if this property is verified (as shown in the diagram below) we say that the two
orthogonal Latin squares are orthogonals:
As you can notice, the Graeco-Latin square design allows the analysis of 4 factors (row, col-
umn, Latin letter, Greek letter), each factor having K levels for only K ⊗ K or K2
possible
combinations.
For the need of the analysis of variance, we have to define certain variables:
• K the number of levels for each factor
• N = K2
, le nombre d’observations yi,j,k,l total dans plan carré gréco-latin, où i désigne le
ie
niveau du facteur représenté en ligne, j le je
niveau du facteur représenté en colonne,
k le ke
niveau du facteur représenté par les lettres latines et l le le
niveau du facteur
représenté par les lettres grecques.
Ainsi, on peut formuler comme suit :
• La somme et la moyenne des observations:
y.... =
K
i=1
K
j=1
K
k=1
K
l=1
yijkl =
K
i=1
K
j=1
yij.. ¯y.... =
y....
K2
(4.7.1208)
• La somme et la moyenne pour chaque ligne:
yi... =
K
j=1
yij.. ¯yi... =
yi...
K
(4.7.1209)
• La somme et la moyenne pour chaque colonne:
y.j.. =
K
i=1
yij.. ¯y.j.. =
y.j..
K
(4.7.1210)
• La somme et la moyenne pour chaque lettre latine:
y..k. =
K
i=1
K
j=1
yijk. ¯y..k. =
y..k.
K
(4.7.1211)

Draft
• La somme et la moyenne pour chaque lettre grecque:
y...l =
K
i=1
K
j=1
yij.l ¯y...l =
y...l
K
(4.7.1212)
Dans la suite, nous allons préférer cette notation K
i,j,k,l à celle-ci K
i=1
K
j=1
K
k=1
K
l=1 yijkl.
Décomposition de la variance totale
Nous allons démontrer que la variance totale :
K
i=1
K
j=1
K
k=1
K
l=1
(yijkl − ¯y....)2
=
K
i,j,k,l
(yijkl − ¯y....)2
(4.7.1213)
peut être écrite:
K
i,j,k,l
(yijkl − ¯y....)2
=
K
i,j,k,l
(¯yi... − ¯y....)2
+
K
i,j,k,l
(¯y.j.. − ¯y....)2
+
K
i,j,k,l
(¯y..k. − ¯y....)2
+
K
i,j,k,l
(¯y...l − ¯y....)2
+
K
i,j,k,l
(yijkl − ¯yi... − ¯y.j.. − ¯y..k. − ¯y...l + 3¯y....)2
Commençons par poser :
K
i,j,k,l
(yijkl − ¯y....)2
=
K
i,j,k,l
(yijkl − ¯yi... + ¯yi... − ¯y.j.. + ¯y.j.. − ¯y..k. + ¯y..k. − ¯y...l + ¯y...l − ¯y....)2
(4.7.1214)
intercalons comme ceci
=
K
i,j,k,l
(yijkl − ¯yi... − ¯y.j.. − ¯y..k. − ¯y...l + 3¯y.... + ¯yi... − ¯y.... + ¯y.j.. − ¯y.... + ¯y..k. − ¯y.... + ¯y...l − ¯y....)2
=
K
i,j,k,l
[(yijkl − ¯yi... − ¯y.j.. − ¯y..k. − ¯y...l + 3¯y....) + (¯yi... − ¯y.... + ¯y.j.. − ¯y.... + ¯y..k. − ¯y.... + ¯y...l − ¯y....)]2
(4.7.1215)
A ce niveau, on retrouve bien notre fameuse identité remarquable (a + b)2
= a2
+ b2
+ 2ab, qui
dans le cadre notre développement, s’applique comme ceci :
=
K
i,j,k,l
(yijkl − ¯yi... − ¯y.j.. − ¯y..k. − ¯y...l + 3¯y....)2
+ (¯yi... + ¯y.j.. + ¯y..k + ¯y...l − 4¯y....)2
+ 2 (¯yi... + ¯y.j.. + ¯y..k + ¯y...l − 4¯y....) (yijkl − ¯yi... − ¯y.j.. − ¯y..k. − ¯y...l + 3¯y....)
=
K
i,j,k,l
(yijkl − ¯yi... − ¯y.j.. − ¯y..k. − ¯y...l + 3¯y....)2
+
K
i,j,k,l
(¯yi... + ¯y.j.. + ¯y..k + ¯y...l − 4¯y....)2
+ 2
K
i,j,k,l
(¯yi... + ¯y.j.. + ¯y..k + ¯y...l − 4¯y....) (yijkl − ¯yi... − ¯y.j.. − ¯y..k. − ¯y...l + 3¯y....)

Draft
Considérons le 3e
membre de cette expression 2 ab :
2
K
i,j,k,l
(¯yi... + ¯y.j.. + ¯y..k + ¯y...l − 4¯y....) (yijkl − ¯yi... − ¯y.j.. − ¯y..k. − ¯y...l + 3¯y....) (4.7.1216)
et démontrons que le résultat de cette dernière est égal à 0. Commençons par le développer
comme ci-dessous :
=
K
i,j,k,l
¯yi... (yijkl − ¯yi... − ¯y.j.. − ¯y..k. − ¯y...l + 3¯y....)(a)
+
K
i,j,k,l
¯y.j.. (yijkl − ¯yi... − ¯y.j.. − ¯y..k. − ¯y...l + 3¯y....)(b)
+
K
i,j,k,l
¯y..k. (yijkl − ¯yi... − ¯y.j.. − ¯y..k. − ¯y...l + 3¯y....)(c)
+
K
i,j,k,l
¯y...l (yijkl − ¯yi... − ¯y.j.. − ¯y..k. − ¯y...l + 3¯y....)(d)
− 4
K
i,j,k,l
¯y.... (yijkl − ¯yi... − ¯y.j.. − ¯y..k. − ¯y...l + 3¯y....)(e)
Développons la première sous-expression (a)
:
K
i,j,k,l
¯yi... (yijkl − ¯yi... − ¯y.j.. − ¯y..k. − ¯y...l + 3¯y....) =
K
i,j,k,l
¯yi... · yijkl −
K
i,j,k,l
¯yi... · ¯yi... −
K
i,j,k,l
¯yi... · ¯y.j..
−
K
i,j,k,l
¯yi... · ¯y..k. −
K
i,j,k,l
¯yi... · ¯y...l + 3
K
i,j,k,l
¯yi... · ¯y....
Évaluons chaque expression issue du développement de (a)
K
i,j,k,l
¯yi... · yijkl =
K
i
¯yi... ·
K
j,k,l
yijkl = K · ¯yi... · ¯yi... = K
K
i
(¯yi...)2
−
K
i,j,k,l
¯yi... · ¯yi... = −K
K
i
(¯yi...)2
−
K
i,j,k,l
¯yi... · ¯y.j.. = −
K
i
¯yi... ·
K
j,k,l
y.j.. = −K¯y.... · K¯y.... = −K2
(¯y....)2
−
K
i,j,k,l
¯yi... · ¯y..k. = . . . = −K2
(¯y....)2
−
K
i,j,k,l
¯yi... · ¯y...l = . . . = −K2
(¯y....)2
+ 3
K
i,j,k,l
¯yi... · ¯y.... = . . . = +3K2
(¯y....)2
(4.7.1217)

Draft
Somme toute nous obtenons que la sous-expression (a)
:
K
i,j,k,l
¯yi... (yijkl − ¯yi... − ¯y.j.. − ¯y..k. − ¯y...l + 3¯y....) (4.7.1218)
est égale à :
K
i,j,k,l
¯yi... · yijkl −
K
i,j,k,l
¯yi... · ¯yi... −
K
i,j,k,l
¯yi... · ¯y.j.. −
K
i,j,k,l
¯yi... · ¯y..k. −
K
i,j,k,l
¯yi... · ¯y...l + 3
K
i,j,k,l
¯yi... · ¯y....
(4.7.1219)
ce qui donne :
K
K
i
(¯yi...)2
− K
K
i
(¯yi...)2
− K2
(¯y....)2
− K2
(¯y....)2
− K2
(¯y....)2
+ 3K2
(¯y....)2
= 0
(4.7.1220)
Ainsi, par le même procédé nous pouvons prouver que chacune des 4 sous-expressions restante
est nulle:
(b)
K
i,j,k,l
¯y.j.. (yijkl − ¯yi... − ¯y.j.. − ¯y..k. − ¯y...l + 3¯y....) =
K
K
j
(¯y.j..)2
− K2
(¯y....)2
− K
K
j
(¯y.j..)2
− K2
(¯y....)2
− K2
(¯y....)2
+ 3K2
(¯y....)2
= 0
(c)
K
i,j,k,l
¯y..k. (yijkl − ¯yi... − ¯y.j.. − ¯y..k. − ¯y...l + 3¯y....) =
K
K
k
(¯y..k.)2
− K2
(¯y....)2
− K
K
k
(¯y..k.)2
− K2
(¯y....)2
− K2
(¯y....)2
+ 3K2
(¯y....)2
= 0
(d)
K
i,j,k,l
¯y...l (yijkl − ¯yi... − ¯y.j.. − ¯y..k. − ¯y...l + 3¯y....) =
K
K
l
(¯y...l)2
− K2
(¯y....)2
− K
K
l
(¯y...l)2
− K2
(¯y....)2
− K2
(¯y....)2
+ 3K2
(¯y....)2
= 0
(e)
− 4
K
i,j,k,l
¯y.... (yijkl − ¯yi... − ¯y.j.. − ¯y..k. − ¯y...l + 3¯y....) =
− 4K2
(¯y....)2
+ 4K2
(¯y....)2
+ 4K2
(¯y....)2
+ 4K2
(¯y....)2
+ 4K2
(¯y...l)2
− 12K2
(¯y....)2
= 0
Maintenant que nous avons prouvé que 2 ab = 0 c’est-à-dire :
2
K
i,j,k,l
(¯yi... + ¯y.j.. + ¯y..k + ¯y...l − 4¯y....) (yijkl − ¯yi... − ¯y.j.. − ¯y..k. − ¯y...l + 3¯y....) = 0
(4.7.1221)

Draft
Ainsi, si l’on remonte à :
K
i,j,k,l
(yijkl − ¯y....)2
=
K
i,j,k,l
(yijkl − ¯yi... − ¯y.j.. − ¯y..k. − ¯y...l + 3¯y....)2
+
K
i,j,k,l
(¯yi... + ¯y.j.. + ¯y..k + ¯y...l − 4¯y....)2
+ 2
K
i,j,k,l
(¯yi... + ¯y.j.. + ¯y..k + ¯y...l − 4¯y....) (yijkl − ¯yi... − ¯y.j.. − ¯y..k. − ¯y...l + 3¯y....)
Vu que le 3e
membre de l’équation, 2 ab est égal à zéro, il ne nous reste que a2
+ b2
:
=
K
i,j,k,l
(yijkl − ¯yi... − ¯y.j.. − ¯y..k. − ¯y...l + 3¯y....)2
+
K
i,j,k,l
(¯yi... + ¯y.j.. + ¯y..k + ¯y...l − 4¯y....)2
(4.7.1222)
Prenons le 2e
membre b2
, de l’expression ci-dessus :
K
i,j,k,l
(¯yi... + ¯y.j.. + ¯y..k + ¯y...l − 4¯y....)2
=
K
i,j,k,l
(¯yi... − ¯y.... + ¯y.j.. − ¯y.... + ¯y..k. − ¯y.... + ¯y...l − ¯y....)2
=
K
i,j,k,l
[(¯yi... − ¯y....) + (¯y.j.. − ¯y....) + (¯y..k. − ¯y....) + (¯y...l − ¯y....)]2
=
K
i,j,k,l
(¯yi... − ¯y....)2
+
K
i,j,k,l
(¯y.j.. − ¯y....)2
+
K
i,j,k,l
(¯y..k. − ¯y....)2
+
K
i,j,k,l
(¯y...l − ¯y....)2
+
K
i,j,k,l
(¯yi... − ¯y....) · (¯yi... + ¯y.j.. + ¯y..k. + ¯y...l − 4¯yi...)
+
K
i,j,k,l
(¯y.j.. − ¯y....) · (¯yi... + ¯y.j.. + ¯y..k. + ¯y...l − 4¯yi...)
+
K
i,j,k,l
(¯y..k. − ¯y....) · (¯yi... + ¯y.j.. + ¯y..k. + ¯y...l − 4¯yi...)
+
K
i,j,k,l
(¯y...l − ¯y....) · (¯yi... + ¯y.j.. + ¯y..k. + ¯y...l − 4¯yi...)
Dans le dernier développement ci-dessus, considérons la sous expression suivante :
K
i,j,k,l
(¯yi... − ¯y....) (¯yi... + ¯y.j.. + ¯y..k. + ¯y...l − 4¯yi...) (4.7.1223)

Draft
et démontrons qu’elle est nulle.
K
i,j,k,l
(¯yi... − ¯y....) · (¯yi... + ¯y.j.. + ¯y..k. + ¯y...l − 4¯yi...)
=
K
i,j,k,l
¯yi... · ¯yi... +
K
i,j,k,l
¯yi... · ¯y.j.. +
K
i,j,k,l
¯yi... · ¯y..k.
+
K
i,j,k,l
¯yi... · ¯y...l −
K
i,j,k,l
¯yi... · 4¯yi... −
K
i,j,k,l
¯y... · ¯yi...
−
K
i,j,k,l
¯y... · ¯y.j.. −
K
i,j,k,l
¯y... · ¯y..k. −
K
i,j,k,l
¯y... · ¯y...l
+
K
i,j,k,l
¯y... · 4¯yi...
= K2
(¯y....)2
+ K2
(¯y....)2
+ K2
(¯y....)2
+ K2
(¯y....)2
− 4K2
(¯y....)2
− K2
(¯y....)2
− K2
(¯y....)2
− K2
(¯y....)2
− K2
(¯y....)2
+ 4K2
(¯y....)2
= 0
Par conséquent, chacune des expressions suivantes :
K
i,j,k,l
(¯y.j.. − ¯y....) · (¯yi... + ¯y.j.. + ¯y..k. + ¯y...l − 4¯yi...) (4.7.1224)
K
i,j,k,l
(¯y..k. − ¯y....) · (¯yi... + ¯y.j.. + ¯y..k. + ¯y...l − 4¯yi...) (4.7.1225)
K
i,j,k,l
(¯y...l − ¯y....) · (¯yi... + ¯y.j.. + ¯y..k. + ¯y...l − 4¯yi...) (4.7.1226)
est égale à 0. Finalement il nous reste :
K
i,j,k,l
(yijkl − ¯y....)2
=
K
i,j,k,l
(¯yi... − ¯y....)2
+
K
i,j,k,l
(¯y.j.. − ¯y....)2
+
K
i,j,k,l
(¯y..k. − ¯y....)2
+
K
i,j,k,l
(¯y...l − ¯y....)2
+
K
i,j,k,l
(yijkl − ¯yi... − ¯y.j.. − ¯y..k. − ¯y...l + 3¯y....)2
=
K
i
K
j,k,l
(¯yi... − ¯y....)2
+
K
j
K
i,k,l
(¯y.j.. − ¯y....)2
+
K
k
K
i,j,l
(¯y..k. − ¯y....)2
+
K
l
K
i,j,k
(¯y...l − ¯y....)2
+
K
i,j,k,l
(yijkl − ¯yi... − ¯y.j.. − ¯y..k. − ¯y...l + 3¯y....)2
(4.7.1227)

Draft
=K
K
i
(¯yi... − ¯y....)2
+ K
K
j
(¯y.j.. − ¯y....)2
+ K
K
k
(¯y..k. − ¯y....)2
+ K
K
l
(¯y...l − ¯y....)2
+
K
i,j,k,l
(yijkl − ¯yi... − ¯y.j.. − ¯y..k. − ¯y...l + 3¯y....)2
(4.7.1228)
Si l’on note SSE la somme des carrés des écarts alors on peut réécrire notre résultat sous cette
forme :
SSET = SSEL + SSEC + SSEl + SSEg + SSEr (4.7.1229)
où:
SSET =
K
i=1
K
j=1
K
k=1
K
l=1
(yijkl − ¯y....)2
(4.7.1230)
SSEL = K
K
i
(¯yi... − ¯y....)2
(4.7.1231)
SSEC = K
K
j
(¯y.j.. − ¯y....)2
(4.7.1232)
SSEl = K
K
k
(¯y..k. − ¯y....)2
(4.7.1233)
SSEg = K
K
l
(¯y...l − ¯y....)2
(4.7.1234)
SSEr =
K
i,j,k,l
(yijkl − ¯yi... − ¯y.j.. − ¯y..k. − ¯y...l + 3¯y....)2
= SSET − SSEL − SSEC − SSEl − SSEg
(4.7.1235)
Avec comme degrés de liberté :
SCET = SSEL + SSEC + SSEl + SSEg + SSEr
K2
− 1 = (K − 1) + (K − 1) + (K − 1) + (K − 1) + (K − 1) (K − 3)
Le degré de liberté pour les carrés des écarts résiduels s’obtient en posant :
K2
− 1 − [(K − 1) + (K − 1) + (K − 1) + (K − 1)] = (K − 1) (K − 3)

Draft
4.7.13.10 Cochran C test
The Cochran’s C test has for purpose to verify the homogeneity of variances for several popu-
lations. This is a preliminary or subsequent test (post hoc) helpful test to do before making a
balanced ANOVA (balanced) analysis and is recommended by the standard ISO 5725 (as the
Tukey’s test we will see much further) !
Although the idea of the Cochran’s C test is empirical, it is nevertheless intuitive as are the
definitions of Dixon and Grubbs tests. Why then do we present in this book in details the proof
of the Cochran’s C test when we have mentioned that we would not do this for the test of Grubbs
and Dixon because the approach of these latter was also empirical? The reason is simple in fact:
the test of Grubbs and Dixon require, at least as far as we know, Monte Carlo simulations to
determine the critical values of rejection or acceptance of the null hypothesis, while the critical
value of Cochran’s C test can be obtained relatively easily analytically.
That said ... we define the Cochran’s C test by the ratio:
Cj =
S2
j
N
i=1
S2
i
(4.7.1236)
where the Si are the unbiased variance of the N different sources of data, each composed
each of n samples and the null hypothesis is intuitively the equality of variances against the
alternative hypothesis that one of the variance is too big (that is to say: bad) and dismissed
because considered as an outlier variance.
The ISO 5725 recommends to repeat this test until there is no longer any aberrant variance
(therefore too large AND far from other variances).
To determine the critical value, let us invert the definition of Cochran’s C test and do some basic
algebraic manipulations:
C−1
j =
N
i=1
S2
i
S2
j
=
S2
j +
N
i=1
S2
i − S2
j
S2
j
= 1 +
N
i=1,i=j
S2
i
S2
j
(4.7.1237)
We note that that pretty much that the second term of the last equality looks like a Fisher law.
As the Fisher is not stable by the addition, we should find a way to turn the term:
N
i=1,i=j
S2
i (4.7.1238)
into a single variance. The idea is then relatively simple but still had someone to think about
it... We know the Si are non-biased variances equation with a factor 1/(n − 1). Therefore if the
N samples (levels) are independent, the overall variance is then for by stability of the Normal

Draft
distribution and by taking the traditional notations of the ANOVA:
S2
global−j =
1
N − 1
N
i=1,i=j
S2
i =
1
N − 1
N
i=1,i=j
Qi
n − 1
=
1
(N − 1)(n − 1)
N
i=1,i=j
Qi
=
1
(N − 1)(n − 1)
Qglobal−j =
Qglobal−j
(N − 1)(n − 1)
(4.7.1239)
Therefore:
C−1
j = 1 +
N − 1
N − 1
N
i=1
S2
i
S2
j
= 1 + (N − 1)
1
N − 1
N
i=1
S2
i
S2
j
= 1 + (N − 1)
Qglobal−j
(N − 1)(n − 1)
Qj
n − 1
(4.7.1240)
We recognize in the last equality the ratio of two squared variances. We then have identically
to what we’ve proved in our study of the one-way fixed factor ANOVA without replication:
Fn−1,(N−1)(n−1) =
S2
j
S2
global−j
=
Qj
n − 1
Qglobal−j
(N − 1)(n − 1) (4.7.1241)
and therefore it comes:
C−1
j = 1 +
N − 1
Fn−1,(N−1)(n−1)
(4.7.1242)
which is therefore independent of j and therefore the left-tail Cochran’s C test (since by by
definition the Cochran’s ratio should be as small as possible) will have for critical value:
C(α, N, n) = 1 +
N − 1
Fn−1,(N−1)(n−1)
−1
(4.7.1243)
If we consider a test with a significance level of 1 − α (thus corresponding to the cumulative
probability of not making a Type I error) and we reiterate it independently again as second time.
Then, if the tests are independent, following the axiom of probabilities, the probability of not
making a Type I error will be given by the product of probabilities:
(1 − α)(1 − α) = (1 − α)2
(4.7.1244)
and so on for n tests. We notice so quickly that the cumulative probability of not making a Type
I error decreases very quickly. For example, for 10 independent repeated tests with a 5% level,
then we have:
(1 − α)n
= (1 − 5%)10 ∼= 59% (4.7.1245)
which is catastrophic! So if we want a level of confidence on repeated tests of a certain value
which we will denote αr, it is clear that we must solve the following equation:
(1 − α)n
= (1 − αr) (4.7.1246)

Draft
Therefore (relationship sometimes named Šidàk equation):
α = 1 − (1 − αr)1/n
= 1 −
1
(1 − αr)n
(4.7.1247)
and with a Taylor expansion to the second order it comes (see section Sequences and Series):
α = 1 − (1 − αr)1/n ∼= 1 − (1 − nαr) = nαr (4.7.1248)
that we name Bonferroni approximation or sometimes also Boole approximation or Dunn
approximation. So in the end, we have for the Cochran’s C test:
C(α, N, n) = 1 +
N − 1
Fα/N,n−1,(N−1)(n−1)
−1
(4.7.1249)
that we can compute with the English version of Microsoft Excel 14.0.6123 using the following
formula:
aIG@IC@xEIAGpsx†@ev€reGxYnEIY@xEIAB@nEIAAA
4.7.13.11 Adequation Tests (goodness of fit tests)
The goodness of fit (GoF) of a statistical model describes how well it fits a set of observations.
Measures of goodness of fit typically summarize the discrepancy between observed values and
the values expected under the model in question. Such measures can be used in statistical
hypothesis testing, e.g. to test for normality of residuals, to test whether two samples are drawn
from identical distributions (see Kolmogorov–Smirnov test further below), or whether outcome
frequencies follow a specified distribution (see Pearson’s chi-squared test below).
4.7.13.11.1 Pearson’s chi-squared GoF test
We will study here our first GoF nonparametric test, certainly one of the most known
and most simple one (which applies only to non-censored data as far as we know).
To introduce this test, assume that a random variable follow a probability distribution P. If we
draw a sample from the population corresponding to this law, the observed distribution, named
sampling distribution, always deviate more or less of the theoretical distribution, taking into
account the sampling fluctuations.
Generally, we do not know the shape of the law P, nor the value of its parameters. It is the
nature of the studied phenomenon and the analysis of the observed distribution that allow to
choose a law likely to be adequate and afterwards to estimate the parameters.
The differences between the theoretical law and the observed distribution can be attributed
either to sampling fluctuations, or to the fact that the phenomenon does not follow, in reality,
the supposed law.
Basically, if the gaps are small enough, we will assume they are due to random fluctuations and
we will accept (not reject in fact!) the supposed law. On the contrary, if the gaps are too high,

Draft
we conclude that they can not be explained solely by the fluctuations and the phenomenon does
not follow the supposed law (reject the null hypothesis).
To assess these gaps and to make a decision, we need:
1. Define the measure of the distance between the empirical distribution and the theoretical
resulting from the retention law.
2. Determine the probability law followed by the random variable giving the distance (in
fact sadly not reject the null hypothesis).
3. State a decision rule to tell from the observed distribution, if the law chosen is acceptable
or not.
First, we will need for this purpose the central limit theorem and secondly recall that during the
construction of the Normal distribution, we have proved that the variable:
x =
k − np
√
npq
=
k − np
np(1 − p)
(4.7.1250)
follow a Normal distribution centered reduced when n approaches infinity (Laplace condition)
and the probability p was very small.
In practice, the approximation is quite acceptable.,. in some companies... and when np 5 and
p ≤ 0.5 therefore (it was one of the terms that needed to tend to zero when we made the proof):
q
√
npq
∼= 0.3 (4.7.1251)
For example in the two figures below where we represented the binomial laws approached by
the Normal associated laws, we have on the left n = 60, p = 1/6, np = 10 and on the right
n = 40, p = 0.05, np = 2:
Figure 4.128 – Approach of binomial functions by associated Normal functions
Finally, remember that we have proved that the sum of squares of n linearly independent re-
duced centered Normal random variable follows a chi-square with n degrees of freedom denoted
by χ2
(n).

Draft
Now consider a random variable X that follows a theoretical distribution function (continuous
or discrete) P and let us draw a sample of size n in the population corresponding to this law P.
The n observations are distributed along k terms (class values) C1, C2, ..., Ck, whose probabil-
ities p1, p2, ..., pk are determined by the distribution function P (refer to the example with the
Henry straight line).
For each modality Ci, the empirical sample size is a binomial random variable ki of parameters:
β(n, pi) (4.7.1252)
This number ki corresponds indeed to the number of successful events: result equal to the
modality Ci of probability pi, obtained during the sampling on n items of the experimental
batch (and not in the population of the theoretical law as before!).
We have proved earlier above during our the study of the binomial law that his expected mean
was:
E(ki) = npi (4.7.1253)
represents the expected theoretical sample size of the modality Ci and its variance is given by
(when n is very big and pi very small):
V(ki) = npiqi = npi(1 − pi) ∼= npi (4.7.1254)
Its standard deviation is therefore:
σki
=
√
npi (4.7.1255)
Under these conditions, provided that the modality Ci has a size npi of at least equal to 5, the
reduced centered variable:
Ei =
ki − npi
√
npi
=
ki − µi
σi
(4.7.1256)
between empirical and theoretical sample size can be approximately regarded as a reduced
centered Normal variable as we prove earlier above.
We now define now the variable:
D =
N
i=1
E2
i =
N
i=1
ki − npi
npi
(4.7.1257)
where ki is often named experimental frequency and npi theoretical frequency.
If we take the square it is because that in a simple sum certain terms would cancel by opposing
effects and thus would mask the differences, if we take the sum of the absolute values the
Statistical Table of D would be difficult to build and the test would be not very robust because
of the small gap of distances. The square allows not only to have an easy statistical table of
D which is simple since it is based on a law with a single parameter, as we shall see, and the
square also sufficiently increases the test’s robustness.

Draft
Note that this variable is also sometimes (somewhat unfortunately) denoted by:
D =
N
i=1
ni − ni
ni
(4.7.1258)
or more often:
D =
N
i=1
Oi − Ei
Ei
(4.7.1259)
This variable D, sum of squared variables Ei, provides a measure of a distance between
empirical and theoretical distribution and empirical distribution. Let us note however that this
is not a distance in the usual mathematical sense (topological).
Recall that D can therefore also be written:
D =
N
i=1
ki − npi
npi
=
N
i=1
(ki − µi)2
σ2
i
(4.7.1260)
D is therefore the sum of squares of N reduced centered Normal random variables linked by
the single linear relation:
k1 + k2 + ... + kN = n (4.7.1261)
where n is the sample size. So D follows a chi-square distribution, but with N − 1 degrees
of freedom, so a degree less because of the unique linear relationship between them! Indeed,
recall that the degree of freedom is the number of independent variables in the sum and not just
the number of summed terms.
Therefore:
D = χ2
(N − 1) =
N
i=1
ki − npi
npi
(4.7.1262)
We name this test the non-parametric Chi-square test or Pearson’s Chi-square test or Chi-
square test of adjustment or Karl Pearson’s test or goodness of ﬁt Chi-square test...
Then, the usage is to determine the value of the Chi-square distribution with N − 1 degrees
of freedom with a 5% probability of being exceeded. Thus, in the hypothesis that the studied
phenomenon, follows the theoretical distribution P, so there is a 95% cumulative probability
that the variable D takes a value less than the one given by the Chi-Square distribution.
If the value of the law of Chi-square obtained from the sample is smaller than that corresponding
to 95% of cumulative probability, we do not reject the null hypothesis that the phenomenon
follows the law P.

Draft
Remarks
R1. The fact that the assumption of the law P is accepted does not mean that this
hypothesis is true, merely that the information given by the sample does not allow
to reject it. Similarly, the fact that the assumption of the law P is rejected does not
necessarily mean that this assumption is false but that the information provided by the
sample rather lead to the conclusion that the inadequacy of such a law.
R2. For the variable D to follow a chi-square law, it is necessary that the theoretical
values npi of the different modalities Ci are at least equal to 5, that the sample was drawn
randomly (no correlation) and that the probabilities pi are not close to zero.
This goodness of ﬁt test however suffers from a major issue: it requires to group the measure-
ments in classes Ci and in practice there is no absolute theorem (at least as far se know) to
choose the number of classes (and in full width) excepted the Sturge Rule proved earlier. It is
this reason that make the Chi-2 goodness of ﬁt test is reserved for discrete distributions where
the problem of the choice of classes not arise.
However, we will need to create goodness of tests that do not require the use of classes and we
will see just after ad hoc tools for this purpose (Kolmogorov-Smirnov, or Anderson-Darling to
name only the most important one).
Example:
Suppose that the births in a hospital for a period of time, are as follows:
Day M T W T F S S Total
Observations 120 130 125 128 80 70 75 728
We note that there were a total of 728 births. We ask then the following question: How
many should there be births, in theory, every day if there is no difference between days?
This represents the null hypothesis H0. In fact the null hypothesis states that the differ-
ences between the observed frequencies and conceptual frequencies are relatively small.
We take for granted that if there is no difference there should be the same number of
births each day. Since there are a total of 728 births for 7 days in theory there should be
728/7 = 104 births every day. So now we have the following table:
Day M T W T F S S Total
Observations 120 130 125 128 80 70 75 728
Expected 104 104 104 104 104 104 104 104

Draft
The total of observed frequencies equals the total expected frequencies. The purpose
is therefore to examine the difference between the observed and expected frequencies
(assumed to follow a uniform law in this special case) using the Chi-square relation. In
other words, we do a fit test between an empirical distribution function (observed) and
the uniform distribution function. Then we have:
χ2
=
N
i=1
(Oi − Ei)2
E2
i
=
(120 − 104)2
1042
+
(130 − 104)2
1042
(125 − 104)2
1042
+
(128 − 104)2
1042
+
(80 − 104)2
1042
+
(70 − 104)2
1042
+
(75 − 104)2
1042
=
162
104
+
262
104
+
212
104
+
242
104
+
(−24)2
104
+
(−34)2
104
+
(−29)2
104
=
256
104
+
676
104
+
441
104
+
(576)2
104
+
576
104
+
1156
104
+
841
104
= 2.46 + 6.5 + 4.24 + 5.54 + 5.54 + 11.12 + 8.09 = 43.49
(4.7.1263)
The χ2
is therefore 43.49. As such this number means little. This result should be
interpreted with the help of the table of critical values of the chi2
N . Without using the
table we understand that it is very unlikely that the observed frequency and theoretical
frequency are identical. We accept that there may be some difference (we therefore reject
the null hypothesis H0 in favor of the alternative hypotheses HA).
So do not forget that this test only applies to uncensored data, that is to say for which the
intervals are closed!
4.7.13.11.2 Kolmogorov-Smirnov GoF test
In statistics, the Kolmogorov-Smirnov test is also a goodness of fit test based on an em-
pirical distance used to determine whether the distribution of a sample follows a well known
law given by a continuous distribution function (or for comparing two samples and check if
they are dependent or not as similar or not). This test, as well as that of Chi-2 GoF test is only
valid for non-censored data (at least not without correction obtained by numerical simulations).
To introduce this test, we chose the Lilliefors approach that give the possibility to avoid complex
calculations. Furthermore, softwares that provide the Lilliefors GoF test do not offer the
Kolmogorov-Smirnov test since the latter is correct only asymptotically (which is the case of
the software Tanagra 4.14).
Imagine we want to build a nonparametric GoF test who works for both discrete and continuous
laws without suffering the same problem as the Chi-square GoF test (clustring in classes).
To build this test, we start from the empirical distribution function already defined at the begin-

Draft
ning of this section and given for reminder by:
ˆFn(x) =
1
n
n
i=1
1xi≤x =
1
n
n
i=1
δxi≤x =
#(i : xi ≤ x)
n
(4.7.1264)
that obviously belong to the interval [0, 1].
Let us now denote by F(x), the true supposed law which analytical expression is known and
with which we would like to compare ˆF(x) and build the distance:
Dn(x) = ˆFn(x) − F(x) (4.7.1265)
Remark
It seems that the letters to represent numbers were used for the first time by Viète in the
middle of the 16th century (but the notation of exposants did not exist at this time).
The reference distribution may, however, also originate from another measurement sample. The
idea is then simply to compare two empirical distributions. We speak then of Kolmogorov-
Smirnov test for 2 independent samples. Some softwares also manage empirically the case
where the two samples do not have the same size.
The problem with this choice of distance is ... what x should we then choose to make a test?
Well to answer it is simple to see that it would be foolish to take a x for which this distance is
minimal, because have a Dn that can be almost zero does not add much information... There-
fore, we rather postponed towards the greatest absolute deviation. Which brings us to redefine
the distance Dn as follows:
Dn(x) = sup ˆFn(x) − F(x) = sup
1
n
n
i=1
1xi≤x − F(x) (4.7.1266)
where Dn is named Kolmogorov-Smirnov empirical distribution (of good course we should
prove rigorously that it is really a distribution ... but for now it is too complex in terms of
the content of this book however this can be verified by numerical simulations). Before going
further with respect to the theory, let’s look at a practical example (because the example is long
we will not put it into the conventional box).
Suppose we measured the following five values:
−1.2, 0.2, −0.6, 0.8, −1.0 (4.7.1267)
thus ordered:
x1 = −1.2, x2 = −1.0, x3 = −0.6, x4 = 0.2, x5 = 0.8 (4.7.1268)
We want to test the following null hypothesis:
H0 : ˆF(x) = Φ(x) (4.7.1269)
where Φ(x) is as usual the distribution function of the Normal centered reduced distribution.

Draft
The empirical distribution function will be given by:
ˆF(x) =



0 =
0
n
=
0
5
for x −1.2
0.2 =
1
n
=
1
5
for −1.2 x −1
0.4 =
2
n
=
2
5
for −1.0 x −0.6
0.6 =
3
n
=
3
5
for −0.6 x +0.2
0.8 =
4
n
=
4
5
for +0.2 x +0.8
1 =
5
n
=
5
5
for x +0.8
(4.7.1270)
(4.7.1271)
Then we traditionally build the following table:
x ˆF(x) Φ(x) | ˆF(x) − Φ(x)|
−1.2−
0 0.115 0.115
−1.2 0.2 0.115 0.085
−1.0−
0.2 0.159 0.041
−1.0 0.4 0.159 0.241
−0.6−
0.4 0.274 0.126
−0.6 0.6 0.274 0.326
+0.2−
0.6 0.580 0.020
+0.2 0.8 0.580 0.220
+0.8−
0.8 0.788 0.012
+0.8 1 0.788 0.212
Often associated with the following graph comparing empirical (in red) and theoretical (in blue)
distributions:
Figure 4.129 – Representation of the approach of the Kolmogorov-Smirnov GoF

Draft
We then observed that the maximum deviation is 0.326. We will denote for after:
d5 = 0.326 (4.7.1272)
Thus some softwares such as Minitab or R denote with the abbreviation KS.
The reader will have noticed that the biggest deviation above the curve is measured by:
D+
n =
n
sup
i=1
i
n
− F(xi) =
n
sup
i=1
i
n
− yi (4.7.1273)
The largest deviation below the curve is measured by:
D+
n =
n
sup
i=1
F(xi) −
i − 1
n
=
n
sup
i=1
yi −
i − 1
n
(4.7.1274)
The biggest deviation is then:
Dn =
n
sup
i=1
D−
n , D+
n =
n
sup
i=1
i
n
− yi, yi −
i − 1
n
(4.7.1275)
But what can we do with this value? To what can we compare it? Well the idea is relatively
simple and involves generating n values (thus 5 in our example) from the distribution law F(x)
of the null hypothesis H0 and compare them to themselves. In other words, to make a Monte
Carlo simulation (see section Numerical Methods).
Thus, in our example, we generate 5 random value values of (0, 1) which gives us example with
Microsoft Excel 11.8346:
axy‚wFƒFsx†@‚exhfi„‡iix@HYIHHHHHHAGIHHHHHHA
We obtain then 5 values of Z (remember it is the usual notation of a random variable of a
Normal centered reduced distribution ) which ordered will be for example:
−1.427, 0.082, 0.162, 0.294, 1.292 (4.7.1276)
and we repeat the same table as before:
x ˆF(x) Φ(x) | ˆF(x) − Φ(x)|
−1.427−
0 0.077 0.115
−1.427 0.2 0.077 0.085
+0.082−
0.2 0.533 0.041
+0.082 0.4 0.533 0.241
+0.162−
0.4 0.564 0.126
+0.162 0.6 0.564 0.326
+0.294−
0.6 0.616 0.020
+0.294 0.8 0.616 0.220
+1.292−
0.8 0.902 0.012
+1.292 1 0.902 0.212

Draft
And so we have the maximum deviation that is 0.333. Either with Microsoft Excel 14.0.6123:
Figure 4.130 – Calculations in Microsoft Excel 14.0.6123
with the explicit formulas:
Figure 4.131 – Explicit calculations in Microsoft Excel 14.0.6123
with the small corresponding VBA routine quickly and poorly made that will take the number of
iterations required in the cell K1 and will put the empirical distribution of Kolmogorov-Smirnov
in column G of the active sheet:

Draft
Figure 4.132 – Small VBA script for K-S GoF test
We therefore reiterate the procedure a thousand times and we get the following distribution
function (obtained simply by making a scatter chart type in Microsoft Excel 14.0.6123 with
2, 000 simulations):
Figure 4.133 – K-S GoF distributin simulation

Draft
and applying a one-sided test with a threshold of α = 5% we get for the 95th percentile:
D5,α
∼= 0.564 (4.7.1277)
The reader will find the same value in the Kolmogorov-Smirnov tables available in many books.
A few thousand simulations are therefore sufficient to restore the values of the tables!
And now, we compare:
(D5,α
∼= 0.564) (d0
∼= 0.326) (4.7.1278)
and therefore we do not reject the null hypothesis.
But ... we must still take care with only five values, it is quite likely that the null hypothesis H0
is not rejected for other distribution laws that the Normal distribution.
Thus, as the reader will have noticed, for each null hypothesis H0 associated with a particular
distribution law, we must tabulate the empirical distribution of Kolmogorov-Smirnov for differ-
ent values of n and of α using numerical methods. In the majority of books there is only one
table with a powerful theorem that shows that in reality, the critical values will be the same.
Remark
Kolmogorov and Smirnov have proved that when n goes very large and that the law of
the null hypothesis is continuous, it is no longer necessary to tabulate a Kolmogorov-
Smirnov table for each law because we have then:
lim
n→+∞
P
√
nDn ≤ x = 1 − 2
+∞
i=1
(−1)k−1
e−2i2x2
(4.7.1279)
therefore Dn is independent of the distribution law of the null hypothesis H0. By simulat-
ing with the Monte Carlo method, we observe an effective convergence when n exceeds
a hundred. But in practice, the vast majority of the time, it is unthinkable to have such
a number of measures. Hence the fact that this theoretical result is little used in practice
and justifies the absence of proof in this book.
To conclude on the K-S GoF test, let us notice to the reader that he will find the mathematical
proof of the Anderson-Darling GoF further below.
4.7.13.11.3 Ryan-Joiner GoF test
Consider a random variable X which we know the sampling distribution and for which
we would like to check the Normality or not and consider an ordered random variable Y
generated by a Normal centered reduced distribution. To compare X and Y , we will center X
and order its values in ascending order.
For a same given sample size, if the ordered values of X and Y taken in pairs follow the same
law, a linear regression based on the other should give a fairly close correlation coefficient of 1.

Draft
Taking the definition of the squared correlation coefficient, we have then:
ρ2
X,Y =
cov(X, Y )
V(X)V(Y )
2
(4.7.1280)
Y is assumed to follow a centered reduced Normal distribution. It comes then:
ρ2
X,Y =
cov(X, Y )
V(X)V(Y )
2
=
E(XY ) − E(X)E(Y )
V(X)V(Y )
2
=
E(XY )
V(X)
2
=
(E(XY ))2
V(X)
(4.7.1281)
and if we take the estimator of the correlation coefficient:
R2
X,Y =
1
n
n
i=1
xiyi
2
1
n
n
n=1
x2
i
(4.7.1282)
Therefore after simplification:
R2
X,Y =
1
n
n
i=1
xiyi
2
1
n
n
n=1
x2
i
=
√
n
n
n
i=1
xiyi
2
n
n=1
x2
i
=
n
i=1
yi
√
n
xi
2
n
n=1
x2
i
=
n
i=1
aixi
2
n
n=1
x2
i
(4.7.1283)
This is the Ryan-Joiner approach (implemented in Minitab for example) of the Shapiro-Wilk
test. The results of both tests are very similar. The coefficients ai can be easily obtained using
any spreadsheet software by using a Monte-Carlo simulation (see section Numerical Methods).
If our reader wish it we will detail how to get the coefficients ai with Microsoft Excel for a
given n.
Example:
Consider the 10 measures of column A already sorted in ascending order:
Figure 4.134 – Example of ordered measures, ranks, RJ coefficient and Z-score

Draft
The corresponding formulas are:
Figure 4.135 – Formulas in Microsoft Excel 14.0.6123 of previous screen shot
and therefore we have in a sheet named CoeffMonteCarlo Monte-Carlo simulations to
determine the 10 coefficients ai traditionally denoted in the case of 10 measurements in
the tables as following: a1/10, a2/10, ..., a10/10 . First we must create 10 columns with
Normal centered reduces variables on almost 10, 000 rows with the following Microsoft
Excel formula:
Figure 4.136 – Centered reduced normal variables for RJ coefficients
and then you have to build the ranks of all these values row by row such as:
axy‚wFƒFsx†@‚exhfi„‡iix@IYWWWWWWWWAGIHHHHHHHHA
and then we have to build the ranks (ascending order) of all these values row by row such
as:
Figure 4.137 – Ascending row by row sorting of simulations to determine the RJ coefficients

Draft
with the following formulas (given only for the first four i because of lack of space on
the screenshot):
Figure 4.138 – Microsoft Excel 14.0.6123 formulas of previous screenshot
Finally, it only remains to calculate the correlation coefficient between the columns E
and E of the first screenshot:
Figure 4.139 – Final calculation of the RJ correlation coefficient with Microsoft Excel 14.0.6123
That gives:
where the square of this value is very very close to the Shapiro-Wilk test.
Then, to know if we reject or not the null hypothesis Ho of Normality assumption,
we should repeat the same procedure with in place of the measurements use random
values generated from a Normal distribution and then determine the critical value of
acceptance/rejection (it is very simple to make but we can detail on readers request).
After calculations in this special case we sadly do not reject the null hypothesis.

Draft
4.7.13.11.4 Anderson-Darling GoF test
It is surprising that a test reasonably strong (robust) as the Kolmogorov-Smirnov test
can be designed based on only a single observation and a single point of the function of
candidate distribution!!! It would seem, with more hindsight, more efﬁcient to measure the
difference between the two distribution functions by comparing these functions on their entire
domain, that is to say from −∞ to +∞.
It exist a family of tests which the statistics are based on the integral of the square of the
difference (these tests are often considered as nonparametric but in my opinion wrongly and
same for the Kolmogorov-Smirnov GoF test that is also considered as nonparametric):
ˆFn(x) − Fn(x)
2
(4.7.1284)
between the empirical distribution function and the reference distribution function. The simplest
of these statistics is:
S =
+∞ˆ
−∞
ˆFn(x) − Fn(x)
2
dx (4.7.1285)
which is simply the area between the empirical distribution function and the reference distri-
bution function. Either, by taking the above graph used during our study of the Kolmogorov-
Smirnov GoF:
Figure 4.140 – Idea behing the Anderson-Darling GoF
However, arbitrarily, we can choose something other than the measurement x for the integral.
Thus, a conventional choice is to take the theoretical distribution function itself as a basic mea-
sure of the integral. It comes as follows:
W2
= n
+∞ˆ
−∞
ˆFn(x) − Fn(x)
2
dF(x) (4.7.1286)
Statistic resulting therefrom is named Cramer-von Mises statistic. However it suffers from a
major ﬂaw of robustness when measuring points are on the tails of the distribution.

Draft
It then was proposed the following measure that is somewhat less sensitive to measurement
points on the tails:
1
n
W2
=
+∞ˆ
−∞
ˆFn(x) − Fn(x)
2
F(x)(1 − F(x))
dF(x) (4.7.1287)
named Anderson-Darling statistical which was the most used in the late 20th century and
remains dominant in the early 21st century also (at least while the sample is a fair size!). It is
more robust by construction that the Cramer-von Mises statistics , and Kolmogorov-Smirnov
simulations, but numerical studies have shown that it is less robust than the Shapiro-Wilk test
or Ryan-Joiner test.
Remembering that the deﬁnition of the empirical distribution ˆFn(x) in our study of the
Kolmogorov-Smirnov GoF test implies

Elements of Applied Mathematics for Engineers

Elements of Applied Mathematics for Engineers

More Related Content

What's hot (19)

Viewers also liked (8)

Similar to Elements of Applied Mathematics for Engineers (20)

Recently uploaded (20)

Elements of Applied Mathematics for Engineers