![5
HABIB’s Complexity 3D Perspective
Eng Emad Farag HABIB
Entropy Meaning
Main Ref.: A. LESNE, “Shannon entropy: a rigorous notion at the crossroads between probability, information theory, dynamical systems and statistical physics”
Understanding the Meaning of Entropy (in Different Sciences): (ref: "Crossroads", Annick Lesne 2011 )
Topic High Entropy String (H=0.9, H=15,..) Low Entropy String (H=0.1,H=0.3,..)
Basics:
Information High Information = Less repeated pattern Low Information = More repeated pattern
Predictability Low Predictability = High Uncertainity High Predictability = Low Uncertainity
Typicality of Disorder Low "Typicality" (High Rarity) Disorder High "Typicality" (Low Rarity) Disorder
Unevenness High Unevenness = Symmetry-breaking Low Unevenness = High-Symmetry
Ex: words like: "aztdn", "odrcr" (from "Wenglish") words like: "ABCDEFGH", ~ "qu….."
IT:
#Digits Large #Digits required to store the info Few #Digits required to store the info
Shannon Entropy (Math.) less Correlated String: Entropy "H" (H~=log2(N) ) Correlated String: "h" (h << log2(N), Dep.)
Indep. Of String-symbols more Independent Sequence more Dep. Seq.[Symbols'-Distr/ Time-Correl.]
Redundancy Scarcely Redundant (Highly distinct) Highly Redundant (scarcely distinct)
%Compressibility Scarcely Compressible (Highly informative) Highly Compressible (scarcely informative)
Missing Info (average) = average I. required to specify the outcome x when the receiver knows the distribution p = amount of uncertainty represented by a pro
Large Missing Information = Large P.Distr. Uncertainity Little Missing Information = Low P.Distr. Uncertainity
Algorithmic Length Large (long) Algorithm to regenerate a String Small (short) Algorithm to regenerate a String
#Ways to (compose) string Few #Ways Many #Ways
Context Uncommon string (within current context) Common string (within current context)
Ex: # : 3.1623 , 3.1103755(another context: √10, π in Octal) # : 4444444, 2468
Probability: [ 1: ELH // 2: P.Distr. : Random Var X, p(x) // 3: Sequences: X, p(x), Types, SubTypes! ]
Uniformity (Elements-wise) more Equal-likelihood Elements less Equal-likelihood Elements
Uniformity (Classes-wise) Similar Classes DisSimilar Classes
Distribution: Event-described ! Distr. Is composed (fully) of Common Events Distr. Is composed (fully) of Rare Events
#States (Possible): TODO Expectation, @states, H,,
Large #: 3(added)dice=4.17 > 1 die=2.58> coin=1 Small #States: coin tossing ( log2|x|=1)
Ex: P. Distr "in/of" string: #Digits to Describe the string
"Normal" (inside 6σ set of values/events) "Normal" (outside 6σ set of values/events)
Dynamical Systems:
#Categories,Elements Large #Categories & Sparse #Elements Few #Categories & Dense #Elements
Ex: Bio. Molecules Protein Structures, Immune-System Cell-Types Simple Structures
VIMP: in Immune System: Healthy: Entropy "booms" @∆ T-Cells & B-Cells ! Eldery ?: minor ∆H: even @large ∆ of Immune threats
Stat. Physics:
( Concerning: Entropy Production "by/via" a dissipative system, rather than Entropy "in/of" the system : Thermodynamic "S" rather than Statistical "H" )
Microstate Molecules: Gas M. are ALL at the same state Molecules: Gas M. are at Different states
Macrostate System: Unable to do useful (mechanical) Work System: Able to do useful (mechanical) Work
Gases Gas in One thermodynamic Compartment Gas in Two thermodynamic Compartments
Ex: P. Distr "by/via" system: S= #Digits of Emergence ! (to Estimate possible Useful work, as opposed to "pure Dissipation")](https://image.slidesharecdn.com/easyperspectivetocomplexity-thriller12slides-220906160303-f3b219bc/85/Updated-version-2-3-THRILLER-Easy-Perspective-to-Complexity-Thriller-12-Slides-pdf-5-320.jpg)
![8
HABIB’s Complexity 3D Perspective
Eng Emad Farag HABIB
A Coherent Perspective to Complexity
Axis X Y (Y-Z Shared !) Z Info Aspects
Axis-Title Orderness Causality (Feedback) Intricacy
System Part
("Scope")
Environ / Sys Sys / Subsys (Inter-Subsys) Subsys / Subsys Info Domains
Main Phenomena Macro Properties, Pattern
formation.
Feedback
(Coded Symbolic).
(Building &
Organizing)
( the SubStr)
Self-Organization
(Subsys, Elements).
Info Usage,
Outcomes
Examples Thermodynamics(PV=
nRT),Fractals, Swarms,
Flocks
Comm: Sampling Rates (2X), mRNA,
Regulatory (Signaling) Pathways?
(Physiology)
(mRNA Vaccines
Marvel! )
Immune Antibodies Diversification
(Germinal Centers)
Info Norms
Quantification Entropy measure: (T.D.,
Shannon)
Hard!, Indirect via: [Non-
Linearity & (Info-)Agents
Formation]
Transfer Entropy ,
…
Measures of: Sophistication,
Hierarchical C., Tree subgraph.
Info
Measures
Main Feature Notion of ~Gestalt Notion of ~Classes Notion of ~Typicality Notion of ~Elements I. Concern
Complexity
Measures
How to Describe the system How to Build the system ( Str / Shared Info) System's Degree of Organization
(Elements-wise).
MIT paper: Info
Measures
Measures
Examples
Information/ Entropy/
Algorithmic Complexity/
Min. Description Length/
Renyi/ Fractal (macro)
Dimension
Logical Depth/ Thermodynamic
D./ Computational Complexity
(,Time, Space)/ Information-
Based C.
(Algorithmic Mutual Info/
Channel Capacity/
Correlation/ Stored Info/
Transfer/ Organization )
Fractal D. (micro!)/ Sophistication/
Effective Measure C./ Hierarchical C./ Tree
Subgraph/ Homogeneous.
MIT Paper by
"Seth Lloyd"
[#3: Str. /
Shared Mutual
Info. ]
~Scale ~macro ~meso (meso-micro) ~micro Info ~Scale
Follows, Guided
by, ..
Simple Rules!
( Statistical)
Communication Rules ( [Speciality/
Numerosity] )
Balance/Duality: [Specifity/
Diversification]
Info "Envelops"
Limits? Spatio-Temporal Limits:
Saturation, Clipping,..
Communication Limits,
Smartness of Agents
( N.A. ! : already
between 2 Extremes)
None!! : Pure Random ! // then
select/elect by -ve Feedback ?
Info
Asymptotes
Info "Types"
(semiotics)
Syntactic (~Form, Objects) Semantic (~Correlations,
relations)
( Learning ) Pragmatic (~Subjective,
Beholder, User)
I. Qualitative
Aspects
Entropy
Concentration
theorems
Sequence space
(Alphabet)
Classes of Sequences (=Types) (Max. Entropy
Distribution? )
Elements (Symbols) I. (Entropy)
Concentration
Comm. Ex. : a "data string" (aggr.) its interpretation its measurement an example
(Action By), the
"Computer"
Sys (not Environ) De-centralized !! (SubSys) De-centralized : just the
(Elements), No "Organizer" !!
Info
Computation
~ ~ Western
Science-Schools
German Science-School:
Constructivism ?
British Science-School:
Empricism ?
American Science-School:
Pragmatism ?
Knowledge
Approach ?
Notes Pattern formation: can be
Scale-free!
VIMP: +veFDBK LIMITS!: e.g. :
Resources, Saturation, Traffic, ..
(Shared Features : can be
considered Y or Z),
~"Transition Features"
Traditional (Classical) Science:
ceases at a Complexity of 3 Elements
!!
Eng. Emad Farag
Habib, Nov 2021
Abbrev.: Information/ System/ Diversification/ Aggregate/ ThermoDynamics/ Feedback/ Complexity (C.) / Communication (Comm.)/ Example/ Not Applicable/ Very Important/ Dimension](https://image.slidesharecdn.com/easyperspectivetocomplexity-thriller12slides-220906160303-f3b219bc/85/Updated-version-2-3-THRILLER-Easy-Perspective-to-Complexity-Thriller-12-Slides-pdf-8-320.jpg)
![9
HABIB’s Complexity 3D Perspective
Eng Emad Farag HABIB
4-Realms :
Probability
Causality
Intricacy
Emergence
DISORDER
+Feedback
(Causality)
+Intricacy
Complex Adaptive Systems
(CAS)
Emergence & Adaptability
( Spontaneous or Equilibrium-based)
==LIMIT ==
+(more ?!)
+1
+FDBK:
Too much -FDBK:
-2 r/n
- r/n
-1,-2, ..
Slight -FDBK
Direct Causality
(non-causal)
ORDER
S0
S1
S0 S1 S2
[ Initial Conditions “External Disturbance” System Adaptation ]
The System responds by “Adapting” to : [ more Intricacy More Info flow More Order ]
Shift from S0 to S1 : Starts Externally (Order), then midway (Causality), then Internally (Intricacy)
Shift from S1 to S2 : Starts Internally (Intricacy), then midway (Causality), then Externally (Order)
S2
CAS System Response
[ More: Order, Links, Intricacy]](https://image.slidesharecdn.com/easyperspectivetocomplexity-thriller12slides-220906160303-f3b219bc/85/Updated-version-2-3-THRILLER-Easy-Perspective-to-Complexity-Thriller-12-Slides-pdf-9-320.jpg)
Updated (version 2.3 THRILLER) Easy Perspective to (Complexity)-Thriller 12 Slides.pdf
- 1. 1 Probability, Causality, Intricacy, and Emergence “Complexity Space” An Easy & Structured Approach to the CONCEPTS of : (Complexity Theory), (Probability & Disorder), (Causality and Feedback) and (Complex Systems) Complexity Theory HABIB’s Complexity 3D Perspective Harmonizing the Concepts of Probability, Causality, Intricacy, and Emergence. Downloadable (for free) for Non-members (and is : Virus, Malignancy, and Macro Free) VERSION 2.3 , September 5th 2022 To get the Latest Version: Open https://www.slideshare.net/EmadfHABIB2/ You will Find ONLY ONE File Named : “UPDATED (Version <whatever>) Easy (Complexity Theory) … “ , While other files are named “Outdated” or have a Completely Different Name (Other Presentations) Eng. Emad Farag HABIB
- 2. 2 ( Quotes ) “Complexity science is so important in today's world .. Many of the most important problems in Engineering, Medicine, and Public Policy are now addressed with the ideas and methods of complexity science.” James Ladyman (University of Bristol), Karoline Wiesner (Universität Potsdam), August 2020 , DOI:10.12987/yale/9780300251104.001.0001 And Author’s book “What is a complex system?” (published with Yale University Press) “Complexity is A MULTI-FACETED Phenomenon, involving a variety of features .. “ ( same a/m authors ) “A variety of Different Measures would be required to capture all our intuitive ideas about what is meant by complexity” The late Physics Nobel Laureate : “MurrayGell-Mann” Complexity Theory HABIB’s Complexity 3D Perspective Harmonizing the Concepts of Probability, Causality, Intricacy, and Emergence. ad Farag HABIB 4-Realms : Probability Causality Intricacy Emergence Importance of Complexity: Complexity “Space” :
- 3. 3 ( Quotes ) “ ... to begin thinking along the LINES of complexity theory. Future Scholars and Scholar-Practitioners will need to think and act Differently when facing Complexity. “ John R. Turner and Rose M. Baker : Complexity Theory An Overview with Potential Applications for the Social Sciences ; doi:10.3390/systems7010004 “Focusing on Information Flow will help us to understand better how cells and organisms work.” Nobel Laureate Paul Nurse Complexity Theory HABIB’s Complexity 3D Perspective Harmonizing the Concepts of Probability, Causality, Intricacy, and Emergence. ad Farag HABIB 4-Realms : Probability Causality Intricacy Emergence Complexity Importance & “Space” : Complexity Core-Issue is “Information Flow” :
- 4. 4 HABIB’s Complexity 3D Perspective Eng Emad Farag HABIB BCR: 4 Cases Distributions: Selection Flowchart Distributions: Curves’ Shapes Distributions: Mathemarical Links
- 5. 5 HABIB’s Complexity 3D Perspective Eng Emad Farag HABIB Entropy Meaning Main Ref.: A. LESNE, “Shannon entropy: a rigorous notion at the crossroads between probability, information theory, dynamical systems and statistical physics” Understanding the Meaning of Entropy (in Different Sciences): (ref: "Crossroads", Annick Lesne 2011 ) Topic High Entropy String (H=0.9, H=15,..) Low Entropy String (H=0.1,H=0.3,..) Basics: Information High Information = Less repeated pattern Low Information = More repeated pattern Predictability Low Predictability = High Uncertainity High Predictability = Low Uncertainity Typicality of Disorder Low "Typicality" (High Rarity) Disorder High "Typicality" (Low Rarity) Disorder Unevenness High Unevenness = Symmetry-breaking Low Unevenness = High-Symmetry Ex: words like: "aztdn", "odrcr" (from "Wenglish") words like: "ABCDEFGH", ~ "qu….." IT: #Digits Large #Digits required to store the info Few #Digits required to store the info Shannon Entropy (Math.) less Correlated String: Entropy "H" (H~=log2(N) ) Correlated String: "h" (h << log2(N), Dep.) Indep. Of String-symbols more Independent Sequence more Dep. Seq.[Symbols'-Distr/ Time-Correl.] Redundancy Scarcely Redundant (Highly distinct) Highly Redundant (scarcely distinct) %Compressibility Scarcely Compressible (Highly informative) Highly Compressible (scarcely informative) Missing Info (average) = average I. required to specify the outcome x when the receiver knows the distribution p = amount of uncertainty represented by a pro Large Missing Information = Large P.Distr. Uncertainity Little Missing Information = Low P.Distr. Uncertainity Algorithmic Length Large (long) Algorithm to regenerate a String Small (short) Algorithm to regenerate a String #Ways to (compose) string Few #Ways Many #Ways Context Uncommon string (within current context) Common string (within current context) Ex: # : 3.1623 , 3.1103755(another context: √10, π in Octal) # : 4444444, 2468 Probability: [ 1: ELH // 2: P.Distr. : Random Var X, p(x) // 3: Sequences: X, p(x), Types, SubTypes! ] Uniformity (Elements-wise) more Equal-likelihood Elements less Equal-likelihood Elements Uniformity (Classes-wise) Similar Classes DisSimilar Classes Distribution: Event-described ! Distr. Is composed (fully) of Common Events Distr. Is composed (fully) of Rare Events #States (Possible): TODO Expectation, @states, H,, Large #: 3(added)dice=4.17 > 1 die=2.58> coin=1 Small #States: coin tossing ( log2|x|=1) Ex: P. Distr "in/of" string: #Digits to Describe the string "Normal" (inside 6σ set of values/events) "Normal" (outside 6σ set of values/events) Dynamical Systems: #Categories,Elements Large #Categories & Sparse #Elements Few #Categories & Dense #Elements Ex: Bio. Molecules Protein Structures, Immune-System Cell-Types Simple Structures VIMP: in Immune System: Healthy: Entropy "booms" @∆ T-Cells & B-Cells ! Eldery ?: minor ∆H: even @large ∆ of Immune threats Stat. Physics: ( Concerning: Entropy Production "by/via" a dissipative system, rather than Entropy "in/of" the system : Thermodynamic "S" rather than Statistical "H" ) Microstate Molecules: Gas M. are ALL at the same state Molecules: Gas M. are at Different states Macrostate System: Unable to do useful (mechanical) Work System: Able to do useful (mechanical) Work Gases Gas in One thermodynamic Compartment Gas in Two thermodynamic Compartments Ex: P. Distr "by/via" system: S= #Digits of Emergence ! (to Estimate possible Useful work, as opposed to "pure Dissipation")
- 6. 6 HABIB’s Complexity 3D Perspective Eng Emad Farag HABIB Appendix E: Links & Interrelations in Systems Dichotomous Classification of Feedback Enough research done in this regard ? Information Correlated Uncorrelated Dependent Independent Flow of Information No Flow Directed Flow Non-directed flow Predictive (Extrapolative) Non-predictive (Non-extrapolative) Transfer (TE) Non-Transfer Causal Non-causal Circular Causality Direct Causality -FDBK Servo (Follows a Variable setpoint) Regulation ( Follows a Fixed Setpoint) +FDBK
- 7. 7 HABIB’s Complexity 3D Perspective Eng Emad Farag HABIB Complexity Space (A Coherent Perspective) Viewing Complexity as a 3D Information Space (# 2 of 4: Complexity Measures : Types & Examples) Axis X Y Z Axis-Title Orderness Causality (Feedback) Intricacy System Part ("Scope") Environ / Sys Sys / Subsys Subsys / Subsys Complexity Measures How to Describe the system How to Build the system System's Degree of Organization (Elements-wise). Measures Examples Information/ Entropy/ Algorithmic Complexity/ Min. Description Length/ Renyi/ Fractal (macro) Dimension Logical Depth/ Thermodynamic D./ Computational Complexity (,Time, Space)/ Information- Based C. Fractal D. (micro!)/ Sophistication/ Effective Measure C./ Hierarchical C./ Tree Subgraph/ Homogeneous.
- 8. 8 HABIB’s Complexity 3D Perspective Eng Emad Farag HABIB A Coherent Perspective to Complexity Axis X Y (Y-Z Shared !) Z Info Aspects Axis-Title Orderness Causality (Feedback) Intricacy System Part ("Scope") Environ / Sys Sys / Subsys (Inter-Subsys) Subsys / Subsys Info Domains Main Phenomena Macro Properties, Pattern formation. Feedback (Coded Symbolic). (Building & Organizing) ( the SubStr) Self-Organization (Subsys, Elements). Info Usage, Outcomes Examples Thermodynamics(PV= nRT),Fractals, Swarms, Flocks Comm: Sampling Rates (2X), mRNA, Regulatory (Signaling) Pathways? (Physiology) (mRNA Vaccines Marvel! ) Immune Antibodies Diversification (Germinal Centers) Info Norms Quantification Entropy measure: (T.D., Shannon) Hard!, Indirect via: [Non- Linearity & (Info-)Agents Formation] Transfer Entropy , … Measures of: Sophistication, Hierarchical C., Tree subgraph. Info Measures Main Feature Notion of ~Gestalt Notion of ~Classes Notion of ~Typicality Notion of ~Elements I. Concern Complexity Measures How to Describe the system How to Build the system ( Str / Shared Info) System's Degree of Organization (Elements-wise). MIT paper: Info Measures Measures Examples Information/ Entropy/ Algorithmic Complexity/ Min. Description Length/ Renyi/ Fractal (macro) Dimension Logical Depth/ Thermodynamic D./ Computational Complexity (,Time, Space)/ Information- Based C. (Algorithmic Mutual Info/ Channel Capacity/ Correlation/ Stored Info/ Transfer/ Organization ) Fractal D. (micro!)/ Sophistication/ Effective Measure C./ Hierarchical C./ Tree Subgraph/ Homogeneous. MIT Paper by "Seth Lloyd" [#3: Str. / Shared Mutual Info. ] ~Scale ~macro ~meso (meso-micro) ~micro Info ~Scale Follows, Guided by, .. Simple Rules! ( Statistical) Communication Rules ( [Speciality/ Numerosity] ) Balance/Duality: [Specifity/ Diversification] Info "Envelops" Limits? Spatio-Temporal Limits: Saturation, Clipping,.. Communication Limits, Smartness of Agents ( N.A. ! : already between 2 Extremes) None!! : Pure Random ! // then select/elect by -ve Feedback ? Info Asymptotes Info "Types" (semiotics) Syntactic (~Form, Objects) Semantic (~Correlations, relations) ( Learning ) Pragmatic (~Subjective, Beholder, User) I. Qualitative Aspects Entropy Concentration theorems Sequence space (Alphabet) Classes of Sequences (=Types) (Max. Entropy Distribution? ) Elements (Symbols) I. (Entropy) Concentration Comm. Ex. : a "data string" (aggr.) its interpretation its measurement an example (Action By), the "Computer" Sys (not Environ) De-centralized !! (SubSys) De-centralized : just the (Elements), No "Organizer" !! Info Computation ~ ~ Western Science-Schools German Science-School: Constructivism ? British Science-School: Empricism ? American Science-School: Pragmatism ? Knowledge Approach ? Notes Pattern formation: can be Scale-free! VIMP: +veFDBK LIMITS!: e.g. : Resources, Saturation, Traffic, .. (Shared Features : can be considered Y or Z), ~"Transition Features" Traditional (Classical) Science: ceases at a Complexity of 3 Elements !! Eng. Emad Farag Habib, Nov 2021 Abbrev.: Information/ System/ Diversification/ Aggregate/ ThermoDynamics/ Feedback/ Complexity (C.) / Communication (Comm.)/ Example/ Not Applicable/ Very Important/ Dimension
- 9. 9 HABIB’s Complexity 3D Perspective Eng Emad Farag HABIB 4-Realms : Probability Causality Intricacy Emergence DISORDER +Feedback (Causality) +Intricacy Complex Adaptive Systems (CAS) Emergence & Adaptability ( Spontaneous or Equilibrium-based) ==LIMIT == +(more ?!) +1 +FDBK: Too much -FDBK: -2 r/n - r/n -1,-2, .. Slight -FDBK Direct Causality (non-causal) ORDER S0 S1 S0 S1 S2 [ Initial Conditions “External Disturbance” System Adaptation ] The System responds by “Adapting” to : [ more Intricacy More Info flow More Order ] Shift from S0 to S1 : Starts Externally (Order), then midway (Causality), then Internally (Intricacy) Shift from S1 to S2 : Starts Internally (Intricacy), then midway (Causality), then Externally (Order) S2 CAS System Response [ More: Order, Links, Intricacy]
- 10. 10 Set of facts: ( regarding Complexity Theory and ultra-Conservative beliefs ) ( Complexity theory is Not the theory of everyhting ) 1- Not Contradicting with beliefs: just Intersecting 2- Intersecting in few issues: that have already settled long ago : Evolution, Randomity, Eventual Thermal Death of the Universe, .. etc. 3- It adds nothing to either sides of the debates or controversies ! : i.e.: those wishing to see Complexity as pro-Evolution: will percieve it so, and those wishing to see Complexity as against-Evolution: will pervieve it so . 4- For a “Non-Scientific Believer” : If such theories causes doubts, they should be skipped and simply left for specialists’. However: for a “Scientific Believer”: Complexity Theory can be of a good & constructive value .. 5- Complexity Theory is NOT pseudoscience. Complexity Theory shares with other sciences the Benefits that ALL Sciences have: that knowledge is good ! , and despite human beings have limited knowledge, Such Knowledge can be developed more, by Studying & Researching, to discover more laws & facts pointing at a Wise-Creator .. Complexity Theory HABIB’s Complexity 3D Perspective Harmonizing the Concepts of Probability, Causality, Intricacy, and Emergence. What Complexity Theory is NOT
- 11. 11 .. and What Complexity Theory is Ref: John R. Turner and Rose M. Baker : “Complexity Theory: An Overview with Potential Applications for the Social Sciences” University of North Texas, 2019 doi:10.3390/systems7010004 Complexity Theory HABIB’s Complexity 3D Perspective Harmonizing the Concepts of Probability, Causality, Intricacy, and Emergence.
- 12. 12 Ver 0.1 20211207 (but was deleted from internet !!!) Ver 0.2 20211209 Ver 0.3 20211210 Ver 0.4 20211212 Ver 0.5 20211215 Ver 0.6 20211218 Ver 0.7 20211222 Ver 0.9 20211224 Ver 1.0 20220101 Ver 1.01 20220103 (Online January 2nd 2022) Ver 1.2 20220104 Ver 1.3 20220106 Ver 1.4 20220114 (Online January 13th 2022) Ver 1.5 20220115 VERSION 1.6 20220123 VERSION 1.7 20220124(Online February 10th 2022) VERSION 1.8 20220211(Online February 11th 2022) VERSION 1.9 20220216 Ver 2.0 20220222 VERSION 2.1 20220301 VERSION 2.2 20220312 ( PDF Thriller 12 Slides only: 20220320) VERSION 2.3 20220905 ( PDF Thriller 12 Slides only: 20220905) Your CRITICISM is Highly Required and any REQUEST of the source file (Concepts file “ XLS” ) is also welcomed : SystemThinking@Inbox.LV ( You can revisit the “Conclusion” Slide ) This Presentation is a draft, will be updated and uploaded later. ( Draft Presentation: due to Author’s Suffering & the need for a Scientific Research Funding or Grant To continue researching such Subject )