Complexity, Big Data Science, and Happiness Discrete Days, St. Michael’s College, 2011

Complexity, Big
Data Science, and
Happiness
Complexity, Big Data Science, and
Happiness
Discrete Days, St. Michael’s College, 2011
Complexity
Introduction
Emergence
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Peter Dodds
Measuring
Happiness
Tweetage
Mechanical Turk
Department of Mathematics & Statistics
Center for Complex Systems
Vermont Advanced Computing Center
University of Vermont
References
1 of 83
Outline
Complexity
Introduction
Emergence
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data & Complex Networks
Nutshell
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
Emergence
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
References
Measuring Happiness
Tweetage
Mechanical Turk
References
2 of 83
Definitions
A meaningful definition of a Complex System:
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
Emergence
I
Distributed possibly networked system of many
interrelated parts with no centralized control
exhibiting emergent behavior—‘More is Different’ [2]
A few optional features:
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
I
Nonlinear relationships
I
Presence of feedback loops
I
Being open or driven
I
Presence of memory
I
Modular (nested)/multiscale structure
I
Opaque boundaries
References
4 of 83
Complexity, Big
Data Science, and
Happiness
Complexity
Examples of Complex Systems:
Introduction
Emergence
Universality
Symmetry Breaking
I
human societies
I
animal societies
I
cells
I
disease ecologies
I
organisms
I
brains
I
power systems
I
social insects
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
I
weather systems
I
geophysical systems
I
ecosystems
I
the world wide web
I
Mechanical Turk
References
i.e., everything that’s interesting...
5 of 83
Complexity, Big
Data Science, and
Happiness
Complexity
Relevant fields:
I
Physics
I
Economics
Introduction
Emergence
I
Cognitive
Sciences
I
Biology
I
Ecology
I
Sociology
I
I
Psychology
I
Information
Sciences
I
Geociences
I
Geography
I
I
Medical
Sciences
Systems
Engineering
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
I
I
Computer
Science
Mechanical Turk
References
...
i.e., everything that’s interesting...
6 of 83
Complexity Manifesto:
1. Systems are ubiquitous and systems matter.
2. Consequently, much of science is about
understanding how pieces dynamically fit together.
3. 1700 to 2000 = Golden Age of Reductionism.
I
Atoms!, sub-atomic particles, DNA, genes, people, ...
4. Understanding and creating systems (including new
‘atoms’) is the greater part of science and
engineering.
5. Universality: systems with quantitatively different
micro details exhibit qualitatively similar macro
behavior.
6. Computing advances make the Science of
Complexity possible:
6.1 We can measure and record enormous amounts of
data, research areas continue to transition from data
scarce to data rich.
6.2 We can simulate, model, and create complex
systems in extraordinary detail.
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
Emergence
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
References
7 of 83
Data, Data, Everywhere—the Economist, Feb 25, 2010 ()
Big Data Science:
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
I
2013: year traffic on
Internet estimate to reach
2/3 Zettabytes
(1ZB = 103 EB = 106 PB =
109 TB)
Emergence
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
I
I
Exponential growth:
∼ 60% per year.
Large Hadron Collider: 40
TB/second.
I
2016—Large Synoptic
Survey Telescope:
140 TB every 5 days.
I
Facebook: ∼ 100 billion
photos
I
Twitter: ∼ 5 billion tweets
Mechanical Turk
References
8 of 83
No really, that’s a lot of data
Frequency
Frequency
they were first invented (1800–1840, 1840–1880,
and 1880–1920) (7). We tracked the frequency
of each invention in the nth year after it was
invented as compared to its maximum value and
plotted the median of these rescaled trajectories
for each cohort.
The inventions from the earliest cohort
(1800–1840) took over 66 years from invention
contrast, “1973” declined to half its peak by
x10-5
1983,
a lag of only 10 years. We are forgetting
our past faster with each passing year (Fig. 3A).
5
We were curious whether our increasing
tendency to forget the old was accompanied by
0more rapid assimilation of the new (21). We divided a list of 147 inventions into time-resolved
cohorts based on the 40-year interval in which
Frequency (log)
RESEARCH ARTICLE
Big Data—Culturomics:
F
ian frequency
Frequency (log)
E (gray lines; median, thick dark gray line). Five examples are highlighted.
1871
Half life
: 73
yrs
yrs
phase (73 years, red line). Inset: The doubling time and half-life over time.
(F) The median trajectory of the 25 most famous personalities born between
1800 and 1920 in various careers.
e: 4
x10-5
Frequency
g ti
me
:4
yrs
: 73 yrs
5
0
F 3. Cultural turnover is accelerating. (A) We forget: frequency of “1883”
Fig.
D “1910” (green), and “1950” (red). Inset: We forget faster. The half-life
(blue),
of the curves (gray dots) is getting shorter (gray line: moving average). (B) Cultural
adoption is quicker. Median trajectory for three cohorts of inventions from three
different time periods (1800–1840, blue; 1840–1880, green; 1880–1920,
red). Inset: The telephone (green; date of invention, green arrow) and radio
(blue; date of invention, blue arrow). (C) Fame of various personalities born
between 1920 and 1930. (D) Frequency of the 50 most famous people born in
(E) The median trajectory of the 1865 cohort is characterized by four
http://www.culturomics.org/
()
14 JANUARY 2011 VOL 331 SCIENCE www.sciencemag.org
parameters: (i) initial age of celebrity (34 years old, tick mark); (ii) doubling
time of the subsequent rise to fame (4 years, blue line); (iii) age of peak celebrity
Google
ngram
viewer
(70 years after birth,Books
tick mark), and (iv)
half-life of the post-peak
forgetting ()
tim
Half life
www.sciencemag.org
SCIENCE
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
F
Emergence
Universality
Median frequency
Downloaded from www.sciencemag.org on January 14, 2011
Year of invention
blin
E
B
D
Dou
Frequency
天安
Frequency
Median
frequency
Frequency
Frequency
(log)
“Quantitative analysis of culture using millions of digitized
books” by Michel et al., Science, 2011 [11]
frequency (log)
1883”
alf-life
ultural
m three
–1920,
radio
s born
born in
Downloaded from www.sciencemag.org on January 14
Median
frequency
Frequency
Frequency
Median
frequency
(log)
Frequency
(% of peak value)
Downloaded from www.sciencemag.org on January 14, 2011
Median frequency
(% of peak value)
rc Chagall” in German (red)
ared to English (blue). (B)
ion of Leon Trotsky (blue),
Zinoviev (green), and Lev
enter a regime marked by slower forgetting:
v (red) in Russian texts,
Collective
memory has both a short-term and a
eworthy events
indicated:
long-term
component.
assassination
(blue arrow),
But executed
there have been changes. The amplitude
and Kamenev
of the Purge
plots is(red
rising every year: Precise dates are
ow), the Great
increasingly
There is also a greater fo), and perestroika
(graycommon.
artheTiananpresent. For instance, “1880” declined
The 1976 cus
and on
1989
to halfboth
its peak
value in 1912, a lag of 32 years. In
uare incidents
led to
discussion in English texts
own on the right). Response
989 incident
largely abA is scale
hinese texts (blue,
shown C
D
ft), suggesting government
ip. (D) While the Hollyen were blacklisted (red
t) from U.S. movie studios,
me declined (median: thick
). None of them were creda film until 1960’s (aptly
Exodus. (E) Artists and writrious disciplines were supby the Nazi regime (red
). In contrast, the Nazis themhick red line) exhibited a
ame peak during the war
) Distribution of suppresces for both English (blue)
man (red) forFthe period from E
945. ThreeC
victims of Nazi
ion are highlighted at left
ows). Inset: Calculation of
pression index for “Henri
.
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
1871 (gray lines; median,
thick dark gray line). Five ex
References
(E) The median trajectory of the 1865 cohort is
parameters: (i) initial age of celebrity (34 years old,
time of the subsequent rise to fame (4 years, blue line);
(70 years after birth, tick mark), and (iv) half-life of t
phase (73 years, red line). Inset: The doubling time
(F) The median trajectory of the 25 most famous per
1800 and 1920 in various careers.
VOL 331
14 JANUARY 2011
10 of 83
Complexity, Big
Data Science, and
Happiness
Homo narrativus:
Complexity
Introduction
Emergence
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
I
I
Mechanisms =
Evolution equations,
algorithms, stories, ...
Rollover zing: “Also, all
financial analysis. And,
more directly, D&D.”
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
References
http://xkcd.com/904/ ()
11 of 83
Basic Science ' Describe + Explain:
Lord Kelvin (possibly):
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
Emergence
Universality
I
“To measure is to know.”
Symmetry Breaking
I
“If you cannot measure it, you
cannot improve it.”
Revolution: Big Data &
Complex Networks
The Big Theory
Nutshell
Measuring
Happiness
Tweetage
Bonus:
I
“X-rays will prove to be a
hoax.”
I
“There is nothing new to be
discovered in physics now, All
that remains is more and
more precise measurement.”
Mechanical Turk
References
12 of 83
Emergence:
Tornadoes, financial collapses, human emotion aren’t
found in water molecules, dollar bills, or carbon atoms.
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
Emergence
Examples:
Universality
Symmetry Breaking
The Big Theory
I
Fundamental particles → Life, the Universe, and
Everything
I
Genes → Organisms
I
Brains → Thoughts
I
People → The Web
I
People → Religion
I
People → Language, and rules in language (e.g.,
-ed, -s).
I
? → time; ? → gravity; ? → reality.
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
References
“The whole is more than the sum of its parts” –Aristotle
14 of 83
Toast + Capers + Almonds = Something Different:
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
Emergence
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
References
15 of 83
Emergence—Mechanism
Thomas Schelling () (Economist/Nobelist):
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
Emergence
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
I
“Micromotives and
Macrobehavior” [14]
I
I
I
Tweetage
Mechanical Turk
References
Segregation
Wearing hockey helmets
Seating choices
[youtube] ()
16 of 83
Reductionism
I
Complex Systems enthusiasts often decry
reductionist approaches . . .
I
But reductionism seems to be misunderstood.
I
Reductionist techniques can explain weak
emergence (e.g., phase transitions).
I
‘A Miracle Occurs’ explains strong emergence.
I
But: maybe miracle should be interpreted as an
inscrutable yet real mechanism that cannot be simply
described. Gulp.
I
Listen to Steve Strogatz and Hod Lipson (Cornell) in
the last piece on Radiolab’s show ‘Limits’ (51:40):
http://blogs.wnyc.org/radiolab/2010/04/
05/limits/
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
Emergence
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
References
17 of 83
The emergence of taste:
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
I
Molecules → Ingredients → Taste/Nutrition/Health
I
See Michael Pollan’s article on nutritionism () in the
New York Times, January 28, 2007.
Emergence
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
References
nytimes.com
I
See also: bumblebees.
18 of 83
Complexity, Big
Data Science, and
Happiness
Limits to what is possible:
Universality ():
Complexity
Introduction
I
I
The property that the macroscopic aspects of a
system do not depend sensitively on the system’s
details.
Key figure: Leo Kadanoff ().
Emergence
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
Examples:
I
References
The Central Limit Theorem:
P(x; µ, σ)dx = √
1
2πσ
e−(x−µ)
2 /2σ 2
dx .
I
Nature of phase transitions in statistical mechanics.
I
Navier Stokes equation for fluids.
20 of 83
Fluids mechanics
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
I
I
I
I
Fluid mechanics = One of the great successes of
understanding complex systems.
Navier-Stokes equations: micro-macro system
evolution.
The big three: Experiment + Theory + Simulations.
Works for many very different ‘fluids’:
I
I
I
I
I
I
the atmosphere,
oceans,
blood,
galaxies,
the earth’s mantle...
and ball bearings on lattices...?
Emergence
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
References
21 of 83
Lattice gas models
Complexity, Big
Data Science, and
Happiness
Collision rules in 2-d on a hexagonal lattice:
Complexity
Introduction
Emergence
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
References
I
I
I
Lattice matters... Only hexagonal lattice works in 2-d.
No ‘good’ lattice in 3-d.
Upshot: play with ‘particles’ of a system to obtain
new or specific macro behaviours.
22 of 83
Hexagons—Honeycomb: ()
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
Emergence
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
References
I
I
Orchestrated? Or an accident of bees working hard?
See “On Growth and Form” by
D’Arcy Wentworth Thompson (). [16, 17]
23 of 83
Hexagons—Giant’s Causeway: ()
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
Emergence
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
References
http://newdesktopwallpapers.info
24 of 83
Hexagons—Giant’s Causeway: ()
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
Emergence
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
References
http://www.physics.utoronto.ca/
25 of 83
Hexagons run amok:
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
Emergence
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
I
I
Graphene (): single layer of
carbon molecules in a perfect
hexagonal lattice (super strong).
Measuring
Happiness
Tweetage
Mechanical Turk
References
Chicken wire () . . .
26 of 83
Whimsical but great example of real science:
“How Cats Lap: Water Uptake by Felis catus” ()
Reis et al., Science, 2010.
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
Emergence
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
References
Amusing interview here ()
27 of 83
Complexity, Big
Data Science, and
Happiness
Symmetry Breaking
Philip Anderson ()—“More is Different,” Science, 1972 [2]
Complexity
Introduction
Emergence
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
I
I
Argues against idea that
the only real scientists
are those working on
the fundamental laws.
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
References
Symmetry breaking →
different laws/rules at
different scales...
(2006 study → “most creative physicist in the world” ())
29 of 83
Complexity, Big
Data Science, and
Happiness
Symmetry Breaking
“Elementary entities of science X obey the laws of
science Y”
Complexity
Introduction
Emergence
Universality
Symmetry Breaking
The Big Theory
I
X
I
Y
I
solid state or
many-body physics
I
elementary particle
physics
I
chemistry
I
solid state
many-body physics
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
I
molecular biology
I
chemistry
I
cell biology
I
molecular biology
Mechanical Turk
References
..
.
vdots
I
psychology
I
social sciences
I
physiology
I
psychology
30 of 83
Symmetry Breaking
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
Emergence
Universality
Anderson:
[the more we know about] “fundamental laws, the less
relevance they seem to have to the very real problems of
the rest of science.”
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
Scale and complexity thwart the constructionist
hypothesis.
References
Accidents of history and path dependence () matter.
31 of 83
More is different:
http://xkcd.com/435/ ()
Complexity, Big
Data Science, and
Happiness
A real science of complexity:
Complexity
A real theory of everything anything:
Introduction
Emergence
Universality
1. Is not just about the ridiculously small stuff...
2. It’s about the increase of complexity
Symmetry breaking/
Accidents of history
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
vs.
Universality
Tweetage
Mechanical Turk
References
I
Second law of thermodynamics: we’re toast in the
long run.
I
So how likely is the local complexification of structure
we enjoy?
I
How likely are the Big Transitions?
34 of 83
Complexification—the Big Transitions:
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
Emergence
Universality
Symmetry Breaking
I
Big Bang.
I
Big Word.
I
Big Randomness.
I
Big Story.
I
Big
Number.
I
Big Replicate.
I
Big Life.
I
Big God.
I
Big Evolve.
I
Big Make.
I
Big Science.
I
Big Data.
I
Big Information.
I
Big Algorithm.
I
Big Connection.
I
Big Social.
I
Big Awareness.
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
References
35 of 83
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
Emergence
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
References
36 of 83
Complexity, Big
Data Science, and
Happiness
Ancestry:
Complexity
Introduction
From Keith Briggs’s excellent
etymological investigation: ()
Emergence
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
I
Opus reticulatum:
I
A Latin origin?
References
[http://serialconsign.com/2007/11/we-put-net-network]
38 of 83
Complexity, Big
Data Science, and
Happiness
Key Observation:
I
I
I
I
I
Many complex systems
can be viewed as complex networks
of physical or abstract interactions.
Opens door to mathematical and numerical analysis.
Dominant approach of last decade of a
theoretical-physics/stat-mechish/combinatorics
flavor.
Mindboggling amount of work published on complex
networks since 1998...
... largely due to your typical theoretical physicist:
I
Piranha physicus
I
Hunt in packs.
I
Feast on new and interesting ideas
(see chaos, cellular automata, ...)
Complexity
Introduction
Emergence
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
References
39 of 83
More observations
I
But surely networks aren’t new...
I
Graph theory is well established...
Complexity, Big
Data Science, and
Happiness
Complexity
I
Study of social networks started in the 1930’s...
I
So why all this ‘new’ research on networks?
I
Answer (to repeat): Oodles of Easily Accessible
Data.
Introduction
Emergence
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
I
We can now inform (alas) our theories
with a much more measurable reality.∗
I
Crucial observation: Real networks occupy a tiny,
low entropy part of all network space and require
specific attention.
I
A central goal: establish mechanistic explanations.
I
What kinds of dynamics lead to these real networks?
∗
Mechanical Turk
References
If this is upsetting, maybe string theory is for you...
40 of 83
Popularity (according to ISI)
“Collective dynamics of ‘small-world’ networks” [20]
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
Emergence
Universality
I
Watts and Strogatz
Nature, 1998
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
I
≈ 4677 citations (as of January 18, 2011)
I
Over 1100 citations in 2008 alone.
Measuring
Happiness
Tweetage
Mechanical Turk
“Emergence of scaling in random networks” [3]
I
Barabási and Albert
Science, 1999
I
≈ 5270 citations (as of January 18, 2011)
I
Over 1100 citations in 2008 alone.
References
41 of 83
Models
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
Emergence
Universality
1. generalized random networks:
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
I
Arbitrary degree distribution Pk .
I
Wire nodes together randomly.
I
Create ensemble to test deviations from
randomness.
Nutshell
Measuring
Happiness
Tweetage
I
Mechanical Turk
References
Interesting, applicable, rich mathematically, very
important.
42 of 83
Complexity, Big
Data Science, and
Happiness
Models
2. ‘scale-free networks’:
Complexity
I
I
Generative, mechanistic
model
Introduction
Emergence
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Ancestory: Herbert Simon’s
model for Zipf’s law [15]
Measuring
Happiness
I
Preferential attachment
model with growth:
References
I
P[attachment to node i] ∝ kαi .
I
Produces Pk ∼ k −γ when
α = 1.
I
Trickiness: other models
generate skewed degree
distributions.
I
γ = 2.5
hk i = 1.8
N = 150
Introduced by Barabasi and
Albert [3]
Tweetage
Mechanical Turk
43 of 83
Complexity, Big
Data Science, and
Happiness
Models
Complexity
3. small-world networks
Introduction
Emergence
I
Introduced by Watts and
Strogatz [20]
Universality
Symmetry Breaking
The Big Theory
Two scales:
I
local regularity (an individual’s friends know each
other)
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
I
global randomness (shortcuts).
I
Shortcuts allow disease to jump
I
Number of infectives increases
exponentially in time
I
Facilitates synchronization
References
44 of 83
Popularity according to books:
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
Emergence
Universality
Linked: How Everything Is Connected to
Everything Else and What It
Means—Albert-Laszlo Barabási
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
References
Six Degrees: The Science of a Connected
Age—Duncan Watts [19]
45 of 83
More observations
I
Web-scale data sets can be overly exciting.
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
Emergence
Witness:
Universality
Symmetry Breaking
The Big Theory
I
I
I
The End of Theory: The Data Deluge Makes the
Scientific Theory Obsolete (Anderson, Wired) ()
“The Unreasonable Effectiveness of Data,”
Halevy et al. [9]
c.f. Wigner’s “The Unreasonable Effectiveness of
Mathematics in the Natural Sciences” [21]
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
References
But:
I
For scientists, description is only part of the battle.
I
We still need to understand.
46 of 83
Examples
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
Emergence
Universality
What passes for a complex network?
I
I
Complex networks are large (in node number)
Complex networks are sparse (low edge to node
ratio)
I
Complex networks are usually dynamic and evolving
I
Complex networks can be social, economic, natural,
informational, abstract, ...
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
References
47 of 83
Complexity, Big
Data Science, and
Happiness
Examples
Physical networks
I
River networks
I
Neural networks
I
Trees and leaves
I
Blood networks
Complexity
Introduction
Emergence
I
The Internet
Universality
Symmetry Breaking
The Big Theory
I
Road networks
I
Power grids
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
References
I
Distribution (branching) versus redistribution
(cyclical)
48 of 83
Complexity, Big
Data Science, and
Happiness
Examples
Interaction networks
Complexity
Introduction
I
The Blogosphere
I
Biochemical
networks
Emergence
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
I
Gene-protein
networks
I
Food webs: who
eats whom
I
The World Wide
Web (?)
I
Airline networks
I
Call networks
(AT&T)
I
Measuring
Happiness
Tweetage
Mechanical Turk
References
datamining.typepad.com ()
The Media
49 of 83
Dynamic networks: Server security
Serving one html page with an image:
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
Emergence
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
References
I
Map of system calls made by a Linux server running
Apache and Windows server running IIS. Which is
which?
Taken from http://www.visualcomplexity.com ()
50 of 83
Complexity, Big
Data Science, and
Happiness
Examples
Interaction networks:
social networks
Complexity
Introduction
Emergence
Universality
I
Snogging
I
Friendships
I
Acquaintances
I
Boards and
directors
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
I
Organizations
I
twitter.com ()
facebook.com (),
I
Measuring
Happiness
Tweetage
Mechanical Turk
References
(Bearman et al., 2004)
‘Remotely sensed’ by: tweets (open), instant
messaging, Facebook posts, emails, phone logs
(*cough*).
51 of 83
11/14/2006 09:04 PM
Complexity, Big
Data Science, and
Happiness
Examples
Relational networks
I
popular | recent
Consumer purchases ()
login | register | help
(Wal-Mart: ≈ 2.5 petabyte = 2.5 × 1015 bytes)
I
Thesauri: Networks of words generated by meanings
I
Knowledge/Databases/Ideas
i/Main_Page
06
I
del.icio.us
search
Complexity
Introduction
Emergence
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Metadata—Tagging: delicious (), flickr ()
Tweetage
Mechanical Turk
common tags
References
cloud | list
community daily dictionary education
encyclopedia
english free imported info information internet knowledge
learning
news
resources
search
reference
tools
useful
research
web
web2.0
resource
wiki
wikipedia
52 of 83
Clickworthy Science:
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
Emergence
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
References
Bollen et al. [5] ; a higher resolution figure is here ()
53 of 83
A notable feature of large-scale networks:
I
Complexity, Big
Data Science, and
Happiness
Graphical renderings are often just a big mess.
Complexity
Introduction
Emergence
Universality
Symmetry Breaking
⇐ Typical hairball
I
I
number of nodes N = 500
number of edges m = 1000
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
I
I
average degree hk i = 4
References
And even when renderings somehow look good:
“That is a very graphic analogy which aids
understanding wonderfully while being, strictly
speaking, wrong in every possible way”
said Ponder [Stibbons] —Making Money, T. Pratchett.
I
We need to extract digestible, meaningful aspects.
54 of 83
Complexity, Big
Data Science, and
Happiness
Properties
Some key aspects of real complex networks:
Complexity
Introduction
Emergence
I
I
degree
distribution Pk ∗
I
I
assortativity
concurrency
hierarchical
scaling
I
homophily
I
network distances
I
clustering
I
centrality
I
motifs
I
efficiency
I
modularity
I
robustness
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
I
Mechanical Turk
References
Plus coevolution of network structure
and processes on networks.
∗ Degree distribution is the elephant in the room that
we are now all very aware of...
55 of 83
Nutshell:
Overview Key Points:
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
Emergence
Universality
I
The field of complex networks came into existence in
the late 1990s.
I
Explosion of papers and interest since 1998/99.
I
Hardened up much thinking about complex systems.
I
Specific focus on networks that are large-scale,
sparse, natural or man-made, evolving and dynamic,
and (crucially) measurable.
Three main (blurred) categories:
I
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
References
1. Physical (e.g., river networks),
2. Interactional (e.g., social networks),
3. Abstract (e.g., thesauri).
57 of 83
Nutshell:
Overview Key Points (cont.):
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
Emergence
I
I
I
Obvious connections with the vast extant field of
graph theory.
But focus on dynamics is more of a
physics/stat-mech/comp-sci flavor.
Two main areas of focus:
1. Description: Characterizing very large networks
2. Explanation: Micro story → Macro features
I
Some essential structural aspects are understood:
degree distribution, clustering, assortativity, group
structure, overall structure,...
I
Still much work to be done, especially with respect to
dynamics... exciting!
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
References
58 of 83
Bonus materials:
Complexity, Big
Data Science, and
Happiness
Complexity
Graduate Course Websites:
Introduction
Emergence
Universality
I
I
Principles of Complex Systems (), University of Vermont
Complex Networks (), University of Vermont
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Textbooks:
David Easley and Jon Kleinberg (Economics and
Computer Science, Cornell)
“Networks, Crowds, and Markets: Reasoning About a
Highly Connected World” ()
I Mark Newman (Physics, Michigan)
“Networks: An Introduction” ()
I
Tweetage
Mechanical Turk
References
59 of 83
Bonus materials:
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
Review articles:
Emergence
Universality
Symmetry Breaking
I
I
I
S. Boccaletti et al.
“Complex networks: structure and dynamics” [4]
Times cited: 1,028 (as of June 7, 2010)
M. Newman
“The structure and function of complex networks” [12]
Times cited: 2,559 (as of June 7, 2010)
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
References
R. Albert and A.-L. Barabási
“Statistical mechanics of complex networks” [1]
Times cited: 3,995 (as of June 7, 2010)
60 of 83
Bonus materials:
I
Complex Social Networks—F. Vega-Redondo [18]
I
Fractal River Basins: Chance and Self-Organization—I.
Rodríguez-Iturbe and A. Rinaldo [13]
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
Emergence
Universality
Symmetry Breaking
I
Random Graph Dynamics—R. Durette
I
Scale-Free Networks—Guido Caldarelli
I
Evolution and Structure of the Internet: A Statistical
Physics Approach—Romu Pastor-Satorras and
Alessandro Vespignani
I
Complex Graphs and Networks—Fan Chung
I
Social Network Analysis—Stanley Wasserman and
Kathleen Faust
I
Handbook of Graphs and Networks—Eds: Stefan
Bornholdt and H. G. Schuster [6]
I
Evolution of Networks—S. N. Dorogovtsev and J. F. F.
Mendes [8]
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
References
61 of 83
Complexity, Big
Data Science, and
Happiness
The Team:
1. People:
Thanks to ...
Kameron Harris
Isabel Kloumann
Complexity
Catherine Bliss
Chris Danforth
Introduction
Emergence
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
2. Machines:
References
3000 processors + storage at the
Vermont
Jonathan Harris & Sep
Kamvar Advanced Computing
wefeelfine.org
Center
I
I
40 TB of storage in Danforth’s office.
3. Support:
NSF and NASA.
63 of 83
Complexity, Big
Data Science, and
Happiness
Happiness:
Complexity
Introduction
Emergence
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
References
Socrates et al.:
eudaimonia [10]
Bentham:
hedonistic
calculus
Jefferson:
. . . the pursuit of
happiness
64 of 83
Early drafts:
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
Emergence
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
References
65 of 83
Complexity, Big
Data Science, and
Happiness
Twitter—living in the now:
Complexity
0.16
breakfast
0.14
lunch
0.12
dinner
Introduction
Emergence
Universality
Symmetry Breaking
count fraction
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
0.1
Measuring
Happiness
Tweetage
0.08
Mechanical Turk
References
0.06
0.04
0.02
0
0
2
4
6
8
10 12 14 16 18 20 22 24
hour of day (local time)
66 of 83
Twitter—living in the now:
Complexity, Big
Data Science, and
Happiness
Complexity
0.07
Introduction
Emergence
Universality
0.06
Symmetry Breaking
count fraction
The Big Theory
Revolution: Big Data &
Complex Networks
0.05
Nutshell
Measuring
Happiness
0.04
Tweetage
Mechanical Turk
0.03
hungry
References
starving
0.02
food
0.01
0
0
eat
2
4
6
8
10 12 14 16 18 20 22 24
hour of day (local time)
67 of 83
Twitter—living in the now:
0.06
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
Emergence
0.05
Universality
Symmetry Breaking
count (%)
The Big Theory
Revolution: Big Data &
Complex Networks
0.04
Nutshell
Measuring
Happiness
0.03
Tweetage
Mechanical Turk
0.02
References
0.01
0
0
2
4
6
8
10 12 14 16 18 20 22 24
hour of day (local time)
A few words you can’t say on television.
68 of 83
Twitter—overall time series:
average happiness havg
6.4
2008—
12/25
2009—
12/25
2010—
12/25
2011—
6.3
A
12/24 01/01
11/27
6.1
04/12
11/26
02/14
6.2
06/21
12/24
12/24 12/31
02/14
05/09
07/04
01/01
12/31
12/31
04/04
10/31
07/04
06/20
10/31
Monday
Tuesday
Wednesday
Thursday
Friday
Saturday
Sunday
02/14
11/25
10/31
01/01
05/08
04/29
04/24
6
09/29
5.9
04/27
06/25
09/14
02/27
08/06
05/24 06/27
10/26
03/11
700
600
B
500
400
300
word count (x10 7 )
Simpson lexical size N S
05/02
Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May
Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May
3
2
C
1
0
Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May
date
Tref : 7 days before and after (havg =6.00)
Tcomp: Monday, 2008/09/29 (havg =5.95)
Tref : 7 days before and after (havg =5.98)
Tcomp: Friday, 2011/04/29 (havg =6.04)
1−↑bailout
−↑bill
−↑down
−↑failed
−↑not
5
wedding +↑
1
dead −↓
dont −↓
death −↓
beautiful +↑
hate −↓
kiss +↑
prince +↑
5
last −↓
−↑no
−↑fail
−↑fails
−↑blame
+↓love
−↑failure
−↑bad
−↑don’t
−↑against
−↑die
−↑rejected
−↑depression
−↑crisis
10
15
money +↑
Word rank r
Word rank r
weekend +↑
25
+↓won
old −↓
−↑worst
+↓great
−↑panic
+↓awesome
−↑didn’t
+↓google
+↓saturday
billion +↑
−↑hurt
+↓win
cancer −↓
0
10
40
−↑sick
−↑problem
−↑crash
+↓friday
−↑falling
1
10
2
10
45
house +↑
Pr
i=1
−20
0
δhavg,i
live +↑
bad −↓
congrats +↑
amazing +↑
kill −↓
ill −↓
nigga −↓
Text size:
wow +↑
Tref Tcomp
died −↓
35
0
10
+↓lol
couple +↑
3
10
10
−↓
20
Per word average happiness shift δhavg,r (%)
50
+↓
+↑
−↑
−↓
+↓friends
kissed +↑
she +↑
killing −↓
4
−↑
0
ass −↓
hell −↓
gorgeous +↑ Balance:
congratulations
+↑
−68 : +168
2
10
45
References
+↓life
1
10
10
−↑gossip
−↑poor
−10
Mechanical Turk
+↓game
miss −↓
4
10
−100
50
Tweetage
married +↑
40
+↓home
−↑killed
−↑fear
3
10
Nutshell
Measuring
Happiness
+↓chocolate
+↓love
+↓win
25
Text size:
Tref Tcomp
Balance:
−165 : +65
+↓
+↑
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
weekend +↑
friday +↑
party +↑
30
35
Universality
+↓me
+↓good
+↓you
20
+↓sunday
+↓fun
+↓party
+↓game
30
Introduction
Emergence
no −↓
princess +↑
never −↓
shit −↓
killed −↓
not −↓
dress +↑
real +↑
15
20
Complexity
+↓easter
+↓happy
10
Complexity, Big
Data Science, and
Happiness
0
P
r
δhavg,i
−30
−20
i=1
100
+↓haha
−10
0
10
20
30
Per word average happiness shift δhavg,r (%)
70 of 83
Tref : Tuesdays (havg =6.03)
Tcomp: Saturdays (havg =6.06)
love +↑
no −↓
haha +↑
party +↑
fun +↑
saturday +↑
1
5
+↓new
6.08
havg
Universality
hahaha +↑
6.06
20
W
T
F
S
S
M
T
W
T
F
S
S
M
day of week
8
7
Word rank r
6.01
T
The Big Theory
live +↑
die −↓
friends +↑
game +↑
con −↓
movie +↑
cant −↓
6.04
6.02
Symmetry Breaking
−↑bored
−↑drunk
15
6.05
6.03
havg
Introduction
Emergence
−↑last
6.07
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
−↑fight
birthday +↑
+↓google
25
Tweetage
great +↑
sunday +↑
family +↑
beautiful +↑
beach +↑
home +↑
30
Mechanical Turk
References
+↓lunch
6
5
sick −↓
shopping +↑
playing +↑
35
−↑don’t
4
T
Complexity
weekend +↑
not −↓
happy +↑
dont −↓
10
2009−05−21 to 2010−12−31:
Complexity, Big
Data Science, and
Happiness
Text size:
Tcomp
amazing +↑T
ref
bad −↓
awesome +↑
homework −↓
wedding +↑
0
10
W
T
F
S
S
M
T
W
T
F
S
S
M
40
day of week
1
10
−↑hangover
−↑miss
+↓free
2
10
45
3
10
4
10
50
0
P
r
i=1
−10
Balance:
−87 : +187
+↓
+↑
shit −↓
court −↓
nice +↑
won +↑
100
+↓school
δhavg,i
−5
movies +↑
0
−↑
5
Per word average happiness shift δhavg,r (%)
−↓
10
71 of 83
valence
rank
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
.
.
.
word
valence
std dev
twitter
rank
g-books
rank
nyt
rank
lyrics
rank
laughter
happiness
love
happy
laughed
laugh
laughing
excellent
laughs
joy
successful
win
rainbow
smile
won
pleasure
smiled
rainbows
winning
celebration
enjoyed
healthy
music
celebrating
congratulations
weekend
celebrate
comedy
jokes
rich
.
.
.
8.50
8.44
8.42
8.30
8.26
8.22
8.20
8.18
8.18
8.16
8.16
8.12
8.10
8.10
8.10
8.08
8.08
8.06
8.04
8.02
8.02
8.02
8.02
8.00
8.00
8.00
7.98
7.98
7.98
7.98
.
.
.
0.93
0.97
1.11
0.99
1.16
1.37
1.11
1.10
1.16
1.06
1.08
1.08
0.99
1.02
1.22
0.97
1.07
1.36
1.05
1.53
1.53
1.06
1.12
1.14
1.63
1.29
1.15
1.15
0.98
1.32
.
.
.
3600
1853
25
65
3334
1002
1579
1496
3554
988
2176
154
2726
925
810
1497
–
–
1876
3306
1530
1393
132
2550
2246
317
1606
1444
2812
1625
.
.
.
–
2458
317
1372
3542
3998
–
1756
–
2336
1198
3031
–
2666
1167
1526
3537
–
–
–
2908
3200
875
–
–
–
–
–
–
1221
.
.
.
–
–
328
1313
–
4488
–
3155
–
2723
1565
776
–
2898
439
4253
–
–
1426
2762
3502
3292
167
–
–
833
3574
2566
–
1469
.
.
.
1728
1230
23
375
2332
647
1122
–
2856
809
–
694
1723
349
1493
1398
2248
4216
3646
4070
–
4619
374
–
–
2256
2108
–
3808
890
.
.
.
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
Emergence
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
References
73 of 83
valence
rank
.
.
.
10193
10194
10195
10196
10197
10198
10199
10200
10201
10202
10203
10204
10205
10206
10207
10208
10209
10210
10211
10212
10213
10214
10215
10216
10217
10218
10219
10220
10221
10222
word
valence
std dev
twitter
rank
g-books
rank
nyt
rank
lyrics
rank
.
.
.
violence
cruel
cry
failed
sickness
abused
tortured
fatal
killings
murdered
war
kills
jail
terror
die
killing
arrested
deaths
raped
torture
died
kill
killed
cancer
death
murder
terrorism
rape
suicide
terrorist
.
.
.
1.86
1.84
1.84
1.84
1.84
1.83
1.82
1.80
1.80
1.80
1.80
1.78
1.76
1.76
1.74
1.70
1.64
1.64
1.64
1.58
1.56
1.56
1.56
1.54
1.54
1.48
1.48
1.44
1.30
1.30
.
.
.
1.05
1.15
1.28
1.00
1.18
1.31
1.42
1.53
1.54
1.63
1.41
1.23
1.02
1.00
1.19
1.36
1.01
1.14
1.43
1.05
1.20
1.05
1.23
1.07
1.28
1.01
0.91
0.79
0.84
0.91
.
.
.
4299
2963
1028
2645
4735
–
–
–
–
–
468
2459
1642
4625
418
1507
2435
–
–
3175
1223
798
1137
946
509
2762
–
3133
2124
3576
.
.
.
1724
–
3075
1618
–
–
–
4089
–
–
175
–
–
4117
730
4428
4474
–
–
–
866
2727
1603
1884
307
3110
–
–
4707
–
.
.
.
1238
–
–
1276
–
–
–
–
4914
–
291
–
2573
4048
2605
1672
1435
2974
–
–
208
2572
814
796
373
1541
3192
4115
3319
3026
.
.
.
2016
1447
226
2920
3782
4589
4693
3724
–
4796
462
2857
1619
2370
143
998
–
–
4528
3126
826
430
1273
3802
433
1059
–
2977
2107
–
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
Emergence
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
References
74 of 83
std dev
rank
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
.
.
.
word
valence
std dev
twitter
rank
g-books
rank
nyt
rank
lyrics
rank
fE@king
fKKkin
fKKked
pussy
whiskey
slut
cigarettes
fKKk
mortality
cigarette
motherfKKkers
churches
motherfKKking
capitalism
porn
summer
beer
execution
wines
zombies
aids
capitalist
revenge
mcdonalds
beatles
islam
pay
alcohol
muthafKKkin
christ
.
.
.
4.64
3.86
3.56
4.80
5.72
3.57
3.31
4.14
4.38
3.09
2.51
5.70
2.64
5.16
4.18
6.40
5.92
3.10
6.28
4.00
4.28
4.84
3.71
5.98
6.44
4.68
5.30
5.20
3.00
6.16
.
.
.
2.93
2.74
2.71
2.66
2.64
2.63
2.60
2.58
2.55
2.52
2.47
2.46
2.46
2.45
2.43
2.39
2.39
2.39
2.37
2.37
2.35
2.34
2.34
2.33
2.33
2.33
2.32
2.32
2.31
2.31
.
.
.
448
1077
1840
2019
–
–
–
322
–
–
–
–
–
–
1801
896
839
–
–
4708
2983
–
–
3831
3797
–
627
2787
–
2509
.
.
.
–
–
–
–
–
–
–
–
3960
–
–
2281
–
4648
–
1226
4924
2975
–
–
3996
4694
–
–
–
4514
769
2617
–
909
.
.
.
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
721
3960
–
3316
–
1197
–
–
–
–
–
460
3752
–
4238
.
.
.
620
688
904
949
2208
4071
3279
185
–
2678
1466
–
2910
–
–
590
1413
–
–
–
–
–
2766
–
–
–
499
3600
4107
1526
.
.
.
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
Emergence
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
References
75 of 83
Complexity, Big
Data Science, and
Happiness
Positive bias in the English language:
0.15
Complexity
Introduction
Emergence
Universality
0.125
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
0.1
Nutshell
N
Measuring
Happiness
0.075
Tweetage
Mechanical Turk
References
0.05
0.025
0
1
2
3
4
5
6
7
8
9
havg
76 of 83
For more...
I
PSD, KDH, IMK, CAB, and CMD
“Temporal patterns of happiness and information in a
global social network: Hedonometrics and Twitter.”
http://arxiv.org/abs/1101.5120 ()
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
Emergence
Universality
Symmetry Breaking
The Big Theory
I
P. S. Dodds and C. M. Danforth
“Measuring the Happiness of Large-Scale Written
Expression: Songs, Blogs, and Presidents.” [7]
Journal of Happiness Studies, 2009.
I
http://www.uvm.edu/∼pdodds/research/ ()
I
http://www.onehappybird.com ()
I
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
References
“Does a Nation’s Mood Lurk in Its
Songs and Blogs?” by Benedict
Carey
New York Times, August 2009. ()
77 of 83
References I
[1]
R. Albert and A.-L. Barabási.
Statistical mechanics of complex networks.
Rev. Mod. Phys., 74:47–97, 2002. pdf ()
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
Emergence
Universality
Symmetry Breaking
The Big Theory
[2]
P. W. Anderson.
More is different.
Science, 177(4047):393–396, 1972. pdf ()
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
[3]
A.-L. Barabási and R. Albert.
Emergence of scaling in random networks.
Science, 286:509–511, 1999. pdf ()
[4]
S. Boccaletti, V. Latora, Y. Moreno, M. Chavez, and
D.-U. Hwang.
Complex networks: Structure and dynamics.
Physics Reports, 424:175–308, 2006. pdf ()
References
78 of 83
References II
[5]
J. Bollen, H. Van de Sompel, A. Hagberg,
L. Bettencourt, R. Chute, M. A. Rodriguez, and
B. Lyudmila.
Clickstream data yields high-resolution maps of
science.
PLoS ONE, 4:e4803, 2009. pdf ()
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
Emergence
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
[6]
[7]
S. Bornholdt and H. G. Schuster, editors.
Handbook of Graphs and Networks.
Wiley-VCH, Berlin, 2003.
Mechanical Turk
References
P. S. Dodds and C. M. Danforth.
Measuring the happiness of large-scale written
expression: Songs, blogs, and presidents.
Journal of Happiness Studies, 2009.
doi:10.1007/s10902-009-9150-9. pdf ()
79 of 83
References III
[8]
S. N. Dorogovtsev and J. F. F. Mendes.
Evolution of Networks.
Oxford University Press, Oxford, UK, 2003.
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
Emergence
Universality
Symmetry Breaking
[9]
A. Halevy, P. Norvig, and F. Pereira.
The unreasonable effectiveness of data.
IEEE Intelligent Systems, 24:8–12, 2009. pdf ()
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
[10] W. T. Jones.
The Classical Mind.
Harcourt, Brace, Jovanovich, New York, 1970.
Mechanical Turk
References
[11] J.-B. Michel, Y. K. Shen, A. P. Aiden, A. Veres, M. K.
Gray, The Google Books Team, J. P. Pickett,
D. Hoiberg, D. Clancy, P. Norvig, J. Orwant,
S. Pinker, M. A. Nowak, and E. A. Lieberman.
Quantitative analysis of culture using millions of
digitized books.
80 of 83
References IV
Complexity, Big
Data Science, and
Happiness
Science Magazine, 331:176–182, 2011. pdf ()
Complexity
[12] M. E. J. Newman.
The structure and function of complex networks.
SIAM Review, 45(2):167–256, 2003. pdf ()
[13] I. Rodríguez-Iturbe and A. Rinaldo.
Fractal River Basins: Chance and Self-Organization.
Introduction
Emergence
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
Cambridge University Press, Cambrigde, UK, 1997.
References
[14] T. C. Schelling.
Micromotives and Macrobehavior.
Norton, New York, 1978.
[15] H. A. Simon.
On a class of skew distribution functions.
Biometrika, 42:425–440, 1955. pdf ()
81 of 83
References V
[16] D. W. Thompson.
On Growth and From.
Cambridge University Pres, Great Britain, 2nd
edition, 1952.
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
Emergence
Universality
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
[17] D. W. Thompson.
On Growth and Form — Abridged Edition.
Cambridge University Press, Great Britain, 1961.
Measuring
Happiness
Tweetage
Mechanical Turk
References
[18] F. Vega-Redondo.
Complex Social Networks.
Cambridge University Press, 2007.
[19] D. J. Watts.
Six Degrees.
Norton, New York, 2003.
82 of 83
References VI
Complexity, Big
Data Science, and
Happiness
Complexity
Introduction
Emergence
Universality
[20] D. J. Watts and S. J. Strogatz.
Collective dynamics of ‘small-world’ networks.
Nature, 393:440–442, 1998. pdf ()
[21] E. Wigner.
The unreasonable effectivenss of mathematics in the
natural sciences.
Communications on Pure and Applied Mathematics,
13:1–14, 1960. pdf ()
Symmetry Breaking
The Big Theory
Revolution: Big Data &
Complex Networks
Nutshell
Measuring
Happiness
Tweetage
Mechanical Turk
References
83 of 83