![Modularity (see Newman & Girvan and other Newman papers)
GOAL: Assign nodes to communities in order to maximize
quality function Q
NP-Complete [Brandes et al. 2008]
~ enumerate possible partitions
Numerous packages developed/developing
• e.g. igraph library (R, python), NetworkX, Louvain
• Need appropriate null model](https://image.slidesharecdn.com/tuesdaymorninglecture-communitiesoverview-170627194011/85/03-Communities-in-Networks-2017-7-320.jpg)
![ ER degree distribution (binomial/Poisson) is not
a good model for many real-world data sets
Independent edges, constrained to expected
degree sequence same as observed.
Requires Pij = f(ki)f(kj), quickly yielding
g resolution parameter ad hoc (default = 1)
[Reichardt & Bornholdt, PRE 2006;
Lambiotte et al., 2008 & 2015]
Modularity (see Newman & Girvan and other Newman papers)](https://image.slidesharecdn.com/tuesdaymorninglecture-communitiesoverview-170627194011/85/03-Communities-in-Networks-2017-8-320.jpg)
03 Communities in Networks (2017)
- 1. Communities in Networks Peter J. Mucha, UNC–Chapel Hill AGRICULTURE APPROPRIATIONS INTERNATIONAL RELATIONS BUDGET HOUSE ADMINISTRATION ENERGY/COMMERCE FINANCIAL SERVICES VETERANS’ AFFAIRS EDUCATION ARMED SERVICES JUDICIARY RESOURCES RULES SCIENCE SMALL BUSINESS OFFICIAL CONDUCT TRANSPORTATION GOVERNMENT REFORM WAYS AND MEANS INTELLIGENCE HOMELAND SECURITY
- 2. Outline & Acknowledgements 1. What is community detection and why is it useful? 2. How do you calculate communities? – Descriptive: e.g., Modularity – Generative: e.g., Stochastic Block Models 3. Where is community detection going in the future? – If time permits (I’ll leave you slides) Skyler Cranmer, James Fowler, Jeff Henderson, Jim Moody, J.-P. Onnela, Mason Porter Dani Bassett, Kaveri Chaturvedi, Saray Shai, Dane Taylor Natalie Stanley, Mandi Traud, Andrew Waugh, James Wilson Eric Kelsic, Kevin Macon, Thomas Richardson JSMF, UCRF (UNC), ARO, CDC, NICHD, NIDDK, NIGMS, NSF Apologies that this presentation will seriously err on the self-absorbed side. It’s a big field, and I do not promise to cover even a small piece of it here.
- 3. Jim Moody (paraphrased): “I’ve been accused of turning everything into a network.” PJM (in response): “I’m accused of turning everything into a network and a graph partitioning problem.” “Structure Function” Philosophical Disclaimer Images by Aaron Clauset
- 4. Karate Club Example This partition optimizes modularity, which measures the number of intra-community ties (relative to a random model) “If your method doesn’t work on this network, then go home.”
- 5. Karate Club Club “Cris Moore (left) is the inaugural recipient of the Zachary Karate Club Club prize, awarded on behalf of the community by Aric Hagberg (right). (9 May 2013)”
- 6. Community Detection Firehose Overview “Hard/rigid” v. “soft/overlapping” clusters cf. biclustering methods and mathematics of expander graphs A community should describe a “cohesive group”: varying formulations/algorithms • Linkage clustering (average, single), local clustering coefficients, betweeness (geodesic, random walk), spectral, conductance,… Classic approach in CS: Spectral Graph Partitioning • Need to specify number of communities sought Conductance MDL, Infomap, OSLOM, … (many other things I’ve missed) … Stochastic Block Models: generative with in/out probabilities between labeled groups Modularity: a good partition has more total intra-community edge weight than one would expect at random (but according to what model?) “Communities in Networks,” M. A. Porter, J.-P. Onnela & P. J. Mucha, Notices of the American Mathematical Society 56, 1082-97 & 1164-6 (2009). “Community Detection in Graphs,” S. Fortunato, Physics Reports 486, 75-174 (2010). “Community detection in networks: A user guide,” S. Fortunato & D. Hric, Physics Reports 659, 1-44 (2016). “Case studies in network community detection,” S. Shai, N. Stanley, C. Granell, D. Taylor & P. J. Mucha, arXiv:1705.02305.
- 7. Modularity (see Newman & Girvan and other Newman papers) GOAL: Assign nodes to communities in order to maximize quality function Q NP-Complete [Brandes et al. 2008] ~ enumerate possible partitions Numerous packages developed/developing • e.g. igraph library (R, python), NetworkX, Louvain • Need appropriate null model
- 8. ER degree distribution (binomial/Poisson) is not a good model for many real-world data sets Independent edges, constrained to expected degree sequence same as observed. Requires Pij = f(ki)f(kj), quickly yielding g resolution parameter ad hoc (default = 1) [Reichardt & Bornholdt, PRE 2006; Lambiotte et al., 2008 & 2015] Modularity (see Newman & Girvan and other Newman papers)
- 9. Null Models for Modularity Quality Functions Erdős–Rényi (Bernoulli) Newman-Girvan* • Leicht-Newman* (directed) • Barber* (bipartite)
- 10. Louvain Method (Blondel et al., “Fast unfolding of communities in large networks”, 2008)
- 11. Facebook Traud et al., “Comparing community structure to characteristics in online collegiate social networks” (2011) Traud et al., “Social structure of Facebook networks” (2012) Caltech 2005: Colors indicate residential “House” affiliations Purple = Not provided
- 12. Facebook Traud et al., “Comparing community structure to characteristics in online collegiate social networks” (2011) Traud et al., “Social structure of Facebook networks” (2012) Caltech 2005: Colors indicate residential “House” affiliations
- 13. Facebook Traud et al., “Comparing community structure to characteristics in online collegiate social networks” (2011) Traud et al., “Social structure of Facebook networks” (2012) Caltech 2005: Colors indicate residential “House” affiliations Purple = Not provided
- 14. U.S. Congressional Roll Call as a similarity network Waugh et al., “Party polarization in Congress: a network science approach” (2009) AGRICULTURE APPROPRIATIONS INTERNATIONAL RELATIONS BUDGET HOUSE ADMINISTRATION ENERGY/COMMERCE FINANCIAL SERVICES VETERANS’ AFFAIRS EDUCATION ARMED SERVICES JUDICIARY RESOURCES RULES SCIENCE SMALL BUSINESS OFFICIAL CONDUCT TRANSPORTATION GOVERNMENT REFORM WAYS AND MEANS INTELLIGENCE HOMELAND SECURITY Adjacency matrix of similarities is dense and weighted, cf. other typical networks (see committees: weighted but sparse) 85th Senate
- 15. U.S. Congressional Roll Call as a similarity network Waugh et al., “Party polarization in Congress: a network science approach” (2009) 85th Senate 108th Senate
- 16. Moody & Mucha, “Portrait of political party polarization” (2013)
- 17. Parker et al., “Network Analysis Reveals Sex- and Antibiotic Resistance- Associated Antivirulence Targets in Clinical Uropathogens” (2015)
- 18. Parker et al., “Network Analysis Reveals Sex- and Antibiotic Resistance- Associated Antivirulence Targets in Clinical Uropathogens” (2015)
- 19. Software Other great codes to know: http://www.mapequation.org/ https://graph-tool.skewed.de/
- 20. Self loops of weight r as a form of resolution parameter Arenas et al., “Analysis of the structure of complex networks at different resolution levels” (2008) (see also Shai et al., “Case studies in network community detection,” 2017)
- 21. Outline & Summary 1. What is community detection and why is it useful? 2. How do you calculate communities? – Descriptive: e.g., Modularity – Generative: e.g., Stochastic Block Models 3. Where is community detection going in the future? – Probably very little time left (if any!) Networks appear in many disciplines Network representations provide a flexible framework for studying general data types, leveraging methods of social network analysis and network science. Community detection is a powerful tool for exploring and understanding network structures, including multilayer networks. Network structures identify essential features for modeling and understanding data in applications.
- 22. Multilayer Networks OrderedCategorical Mucha et al., “Community structure in time-dependent, multiscale, and multiplex networks” (2010) Kivelä et al., “Multilayer Networks” (2014)
- 23. Multilayer Modularity Mucha et al., “Community structure in time-dependent, multiscale, and multiplex networks” (2010) Generalized Lambiotte et al. (2008) connection between modularity and autocorrelation under Laplacian dynamics to re-derive null models for bipartite (Barber), directed (Leicht-Newman), and signed (Traag et al.) networks, specified in terms of one-step conditional probabilities intra-layer adjacency data and null inter-layer identity arcs Same formalism works for more general multilayer networks, with sum over inter-layer connections within same community
- 25. Bassett et al. “Dynamic reconfiguration of human brain networks during learning” (2011)
- 26. Cranmer et al., “Kantian fractionalization predicts the conflict propensity of the international system” (2015) • Identified communities of nation states in multiplex international relations of trade, IGOs, democracies • Granger causal relationship to total system-level conflict • Negligible contribution from joint democracy layer
- 27. Stanley et al., “Clustering network layers with the strata multilayer stochastic block model” (2016)
- 28. See mapequation.org Phys. Rev. X 6, 011036 (2016)
- 29. Stanley et al., “Clustering network layers with the strata multilayer stochastic block model” (2016)
- 30. Stanley et al., “Clustering network layers with the strata multilayer stochastic block model” (2016) Initialization layer l kmeans cluster L layers in to S strata stratum s Iterative Process stratum s Update number of strata to the number of unique clustering patterns according to (1) and (2) kmeans cluster 2L layers in to S strata (1) (2) ns r L in a stratum s kmeans cluster tion layer l kmeans cluster L layers in to S strata stratum s Process kmeans cluster 2L layers in to S strata (1) (2) tion layer l kmeans cluster L layers in to S strata stratum s Process kmeans cluster 2L (1) kmeans cluster L layers in to S strata stratum s
- 31. Taylor et al., “Enhanced detectability of community structure in multilayer networks through layer aggregation” (2016)
- 32. Taylor et al., “Enhanced detectability of community structure in multilayer networks through layer aggregation” (2016)
- 33. Community Detection Firehose Overview “Hard/rigid” v. “soft/overlapping” clusters cf. biclustering methods and mathematics of expander graphs A community should describe a “cohesive group”: varying formulations/algorithms • Linkage clustering (average, single), local clustering coefficients, betweeness (geodesic, random walk), spectral, conductance,… Classic approach in CS: Spectral Graph Partitioning • Need to specify number of communities sought Conductance MDL, Infomap, OSLOM, … (many other things I’ve missed) … Stochastic Block Models: generative with in/out probabilities between labeled groups Modularity: a good partition has more total intra-community edge weight than one would expect at random (but according to what model?) “Communities in Networks,” M. A. Porter, J.-P. Onnela & P. J. Mucha, Notices of the American Mathematical Society 56, 1082-97 & 1164-6 (2009). “Community Detection in Graphs,” S. Fortunato, Physics Reports 486, 75-174 (2010). “Community detection in networks: A user guide,” S. Fortunato & D. Hric, Physics Reports 659, 1-44 (2016). “Case studies in network community detection,” S. Shai, N. Stanley, C. Granell, D. Taylor & P. J. Mucha, arXiv:1705.02305.
- 34. Outline & Summary 1. What is community detection and why is it useful? 2. How do you calculate communities? – Descriptive: e.g., Modularity – Generative: e.g., Stochastic Block Models 3. Where is community detection going in the future? Networks appear in many disciplines Network representations provide a flexible framework for studying general data types, leveraging methods of social network analysis and network science. Community detection is a powerful tool for exploring and understanding network structures, including multilayer networks. Network structures identify essential features for modeling and understanding data in applications.
- 35. Special thanks to Mucha Research Group 2016–17