Connections between the Lines: Augmenting Social Networks with Text KDD 09

Connections between the Lines:
Augmenting Social Networks with Text
Jonathan Chang, Jordan Boyd-Graber, David M. Blei
Princeton University
Identity Guidelines
The signature
The Princeton signature should be
included on all official Princeton
publications.
Signature or wordmark
SIGNATURE
It need not appear in a large or
prominent position, but it should be
included to signal a publication’s
core relationship to Princeton.
6
KDD 09
June 29th, 2009
The signature should be used in all
situations that call for the Princeton
“logo,” for instance on promotional
materials for public events for which
Princeton is the primary sponsor.
The wordmark
The wordmark may be used alone in
some situations. When the wordmark
is used alone, the shield should also
appear somewhere on the publication
or item. It may be particularly useful to
incorporate the wordmark separately
from the shield in display settings or
in less formal situations.
Digital art for the Princeton signature
and wordmark is available to the
WORDMARK
Jonathan
Chang,
Princeton community
for use on Jordan Boyd-Graber, David M. Blei
Connections between the Lines
Motivation
Data that express relationships between ensembles of entities
(people, places, companies, genes, etc.) is pervasive.
Jonathan Chang, Jordan Boyd-Graber, David M. Blei
Connections between the Lines
Motivation
Annotating the edges of these networks with rich information is
often a desideratum.
Jonathan Chang
Jordan Boyd-Graber
You and Jordan both
went to Princeton.
Jonathan Chang, Jordan Boyd-Graber, David M. Blei
Connections between the Lines
Motivation
Unfortunately, much of the rich information about links is encoded
implicitly as free text.
Example
In 1938, Wyman co-starred with Ronald Reagan. Reagan and actress
Jane Wyman were engaged at the Chicago Theater and married in
Glendale, California. Following arguments about Reagan's political
ambitions, Wyman led for divorce in 1948. Since Reagan is the only
U.S. president to have been divorced, Wyman is the only ex-wife of an
American President.
Jonathan Chang, Jordan Boyd-Graber, David M. Blei
Connections between the Lines
Motivation
Using the text as input, our goal is to annotate a network with this
information.
Example
Ronald Reagan
Jane Wyman
You and Jane used to
be married.
Jonathan Chang, Jordan Boyd-Graber, David M. Blei
Connections between the Lines
Main idea
We dene a probabilistic model Networks Uncovered By
Bayesian Inference (Nubbi).
Using Nubbi, we can create a social network by discovering
relations over the ensemble of people.
Because Nubbi is a fully Bayesian approach, we can
incorporate information from other data-mining approaches as
priors (Agichtein and Gravano 2003; Diehl et al. 2007; Mei
et al. 2007; Sahay et al. 2008; Banko et al. 2007; Katrenko
and Adriaans 2007; Davidov et al. 2007).
Wikipedia
Hubert
Humphrey
republican
state
federalist
vote
vice
Ross
Perot
Lyndon B.
Johnson
Richard
Nixon
Jimmy
Carter
George H. W.
Bush
John F. Kennedy
soviet
russian
govern
union
nuclear
Mikhail
Gorbachev
Ronald
Reagan
Jonathan Chang, Jordan Boyd-Graber, David M. Blei
Nikita
Khrushchev
labour
govern
leader
british
world
Margaret
Thatcher
Joseph
Stalin
Lev
Kamenev
soviet
communist
central
union
full
Winston
Churchill
Mao
Zedong
Connections between the Lines
Leon
Trotsky
Zhou
Enlai
e ls m
ike pp ae llia y m
m se ich wi en
a,
m e, vg
zz i, giu , ar ye y v
er
pia rdkakisespeov, gu
ve ak ak i,
nif rry
du
sh im linarspikei, jen, ga s
pr , iat ov
tie
mo pr ar
ma
lee sp , al tho chris
ee
,
ca
ka ter e, an−yo tonyx sir
lei esitm
l
wi , yoner, no , de
wh
len latrel t
ma sh is, ington
ku sh ell, ren
lau ry
lew
wa rew a, , jersteve m
ny
sp bilwn s, fay
ert rvin en
ka
bro illip nt,, rob, ea antho
ph l
ph ce
i,
vin tesnson sin , stehard k
ga ma im ric rrich
joh
he
de
tomnddsky,an, mas ith i
so
ke
bro lem , tho r, , hosn
co an he
ke ds rakbarrygustglen
bra ba s, au y a
r el−
mu nd n, ne rtin
ma
bo so rodma
wil g, , b am e jr
kin gis bo muboon w
hin le, afi, t nd n
do dd s, mo phoiii
qa ken ray ste l
pic ly, ry, l paue mas
kel rbu r, kob tho
ma me , b
bre ant no, n n
bry isa joh joh dam
gol nn, n, sad
a
gle dde n,
bar h
ma sei , tom
ralp
hus kin s, bar
iel
har lter nthal, l
dan
wa me ted er ra,
blu er, oliv ved
turn th, saa, a m l
nor ga hal pau
orteent ue,
ros liabjay hak l
tag , yitz hae
lenoin, mic y
rab an, andene
poll itte, eug
pett ill, woo as r
o'ne tae thom
roh zzi, tom, nicolae
suo aw, scu ad h jr
brokuse ahm rles
cea labi, cha
cha ing, isiah
in
keat as,
k
thom r, melv
milleris, mar
o m an
mor itt, eric
schm o, mari
ge herm
cuom geori
ruth, ein gary a
huss field, imeld
shef os,
mj
marc , willia
perry karl leon
ia
rove, ein,
rd
botst , c virgin
leonaer m
fields tein, sumn
bernsone, a alfred
ouk
redst an,
taubmom sihan
lee
norod , john s s
malvo charle james e
robb, evey,
mcgre , ariel j
sharonhenry
stern, , robert
moses nd, strom
thurmo thabo
mbeki, ey, brian
mulronwayne h jr
king,
, alton
maddox n, dave
anderso todd s
purdum,
i
klein, joel
john da jr
dingell,
samuel
alito, bill
frist, bill
cosby,elizabeth h
dole,
feld, eliot
hariri, rafik
williams, serena
hart, garyalbert
einstein,
r jr
biden, joseph s
foley, thomas e
shalala, donna
manning, eli
sistani, ali al−
martinez, tino
bonilla, bobby
franks, bob
beckett, samuel
eisner, michael d
abramoff, jack
lewis, anthony
padilla, jose
winerip, michael
edwards, john
john, elton
henderson, rickey
barenboim, daniel
gonzalez, elian
moore, michael
becker, boris
vance, cyrus
pirro, jeanine r
kennedy, edward
m
holtzman,
feinstein, elizabeth
dalai lamadianne
robbins, jerome
warhol, andy
fehr, donald
pitino,
james, rick
gates, lebron
mobutubill
sese seko
ali, muhamm
ad
noriega,
rangel, manuel antonio
kristof, charles b
nicholas
levin, carl
d
wassers
thomps tein, wendy
rostenk on, william
c jr
starr, owski,
shevarkenneth dan
sullivandnadze,w
eduard
,
chirac, louis
a
abram jacque w
martin s, roberts
roth, , steve
izetbephilip
musc govic,
summhamp, alija
herbe
green ers, lawre
rt
wood , richar
nce h
joyce ward, d r
arafa , jame bob
zedil t, yasir s
gretz lo ponc
ward ky, wayne de
mille , benja
e leon, erne
florio r, judith min
sto
dean , jame
east , howas j
chre woo
rd
farra tien, d, clint
falw khan jean
man ell, , louis
geff delajerry
, winn
john en, davi
scot son,
d ie
mag
weic t, byro
mox ker, n ic
pelo ley, lowe
john si, mar ll p
difra sonnancytha jr
free nce , lynd
mus h, sco on
bru sinaloui , don bain
ald es
mill der , s j
t
comer, , thomike
sun bs, arth ma
rice unu seaur s
, joh n
lam ,
lett ontcondol n
con erm , ned eezh
kle ner an,
za
gre in, , dendav
id
pau enhcal
buf lso ousvin nis
ste fett n, e,
cle inb , wahen ste
kan me erg rre ry ven
cha tor, ns, , joe n m
bak vez microg l be jr
bru er, ,
er
bia ni, jamjuliokey
kac gg fra
ces
car zyni, nk es a ar
3d
roh luc ma
kar aty ci,ski rio
kim zai n, fra, the
jian
, feli nk od
gre da ha x c ore
sch eng ze e junmid g
j
we
, mi g
riv lchundlema n
rk
kim era , joh
r, bre
mc
,
bro enjon ma n f t
we
roeg il rianojr d
wa llstdy,
de gn jan , joh
mu lay er,one, e e n
ov
,
tay itzrdochtomrichapaul
,
mo
rd
qu lor mi ,
go ayrge , lawch rup
sc lde nth renael ert
ca ow le,
ho rte cron, hodanau, ce
rob
bli
nu x, ustonr, bil ft, wa
be
br rd er t
ed ss ha , l
en
m
re lki ba ns all
t
bir nobergn, lisum an
ro
ha bind, , jan, a , he
m nk solar etstefan dd
ry
su an
a
pir lliv tles, ton,
ro an , m jac
, jea , m
kie
an ick
nin dr ey
e ew
f
A larger example...
a
on
em
, an x
llis , alem cia
co ez to tri ly
rto igu ard, , pa
ha dr pp an on ato
ro wm jas, an ul
sto d, ov pa l rnie j f jr
bo rp , bil be ew m
kid on n, , dr
llia io
ka
sim nto mso, an, wi rg h
cli llia
gio
wi an leyni, ep iv
y
sp ck , jos raj
nn jr
bu malin hi, n ke n f a
h
ar
sta ndg, dorson,y, joh m
ga
llia ter ret
kin de edl, wi pe rga b n
an nneil s, ma rd
ke rtin er, na nto
o'n tch leo wy
,
ma
x
tha ndrsalis,
sa lcolmphilipte ou
,
ma
ad h
ma ss vicen
am ria n
gla ,
v c iv
fox llo, , mar, joh
dia rey novaclaeph
ca on l, jos
o'c ve n, te hard
ha so pe, ric n
l
wil e,
y hael ard
ros itch , joh
rich
rav rks , vija
sta gh , mich, ri
sin son urg ter
wil rnb o, perpat ine a
tho iav jas ,
sch ns, songerald
joh ert , charlesp m
rob aro ven es
ferr ley,ste jam
oak s, s, iam ine
job ord will her
jeff ire, katjohn n
g
saf ris, n, briad
har cai an, lloy my
mc hm n, , tom
cas tse son r p
ben mp tige rgenis
tho ds, geo
woo ltz, , j den
shu tertrobert
j
has r,
pea isto williamid
chr ey, dav rt ihiro
cas et, robemor
maman, wa,
beth
altm okarudy mary
hos , ad, k
ud
crewehe , mar
ed dao
whit sier by
mes , bobdick on c d rash
holikney, , ram ame
che ines li, mohy b
cort wha harr w
al−'o sley, ge ak
helm , geor
bush ir, yitzh
dore
sham n, ilan
, theos
ramoeveltjohn
us
roos in,
mcca y, dick rd milho
arme , richa
nixonaus, jack ela
nickl tini, gabri
saba g, j k ing av
rowlin xiaop vojisl il s
deng nica, mikha
kostu chev,joe
gorba na,
montalls, bill
parce ross
iro
perot, i, junich
koizum jimmy
carter, g, curtda al−
schillinmokta
sadr, , davidj
brooksporter lynn
goss, , terry
nicholshenry j
hyde, , ted
williams t, florence
fabricanjames
levine, nancy
reagan, e, vinny
corbin
testaverd
jones, paula
henry a
waxman,
blair, tony , bill
richardson
paul e
tsongas, tawana
brawley,edward
albee,
johnson, randy
daly, john
lopez, jennifer
holland, bernard
graham, bob
pareles, jon
brown, edmund g jr
martin, curtis
volcker, paul a
maslin, janet
stern, david
levitt, arthur jr
wilson, pete
whitman, christine todd
pope
poindexter, john m
ellington, duke
courier, jim
bruno, joseph l
hun sen
snow, john w
kemp, jack f
oates,
joyce
carol
levy, steve
waldheim, kurt
strahan, michael
powell, colin l
lewinsky,
nunn, sammonica s
camby, marcus
ozawa, seiji
mcfarlane,
robert c
martin,
riley, patbilly
krugman,
dimaggio, paul
messinge joe
webber, r, ruth w
mickelso chris
savimbi, n, phil
bloombe jonas
da silva, rg, michael
li peng luiz inacio r
lula
charles
, prince
van gogh,
of wales
graham
vincent
maliki, , martha
pierce, nuri kamal
collins samue
al−
canby, , kerry l r jr
badillo vincen
papp, , hermat
prodi, joseph n
weins roman
o
lipsyt tein, harve
o'nea e, rober
y
foste l, shaqu t
pack r, vince ille
spiel wood nt w
kelle berg, , rober jr
bumi r, bill steve t w
picas ller, elisa n
pere so, pablo
beth
steves, shim
bettm ns,
on
thom an, scot
kerr as, garyt
hill, y, johnclarence
milo anita
wein sevi f
reag gart c, slob
belic an, en, oda
scio hick rona randi n
gott lino, , bill ld wils
on
hem i, johnelai
ne
mar ingw
fujimtin, ay,
mur ori, ken erne
kim phy albeyon st
aqu you , rich rto
torr ino, ng ard k
mu icel corasam
leo tom li, rob zon
lind nar bo, ert c
ack ros d, sugdike g
sca erm , eric
ar mbe
ray
ma lia, an,
ridg lon ant felic
torr e, e, oni ia
mc e, tomjohn n
c
col car joe
miy lins tne
spi aza , gley,
bro tze wa nnpaul
gor wn r, ,
bak e, , elliotkiic
gib er, albdav l hi
len son al ert id m
jr
wa no ,
sch lsh n, me
dia rod , joh l
sch na law n
jon les , er, gerence
do es, prince
rha e
so le, inger,
ss rd
as ros elizmario
art of
ma hc ,
ge su rof ge abethn hur wales
m
sim rst r, t, org
jr
sim ne ku joh e
pa ps r, rt n
ke teron on lou
we vo no, ne, o is
do inb rki , il j v jr
lee le, ergan joe
,
gia
ba tch rob er, jac
din rysmatti,, bri ert ca k
do
j sp
ba mikin hn a an
ar
vo ke s, iko ba
w
joh lpe r, ngo, davidv, rtle
mi
hit
ru
ge lernson, jus ss pla n kh tt
pa
ail
ell cid
me rsh , ad , tin
o
no tak wi olflarry a
llo es i, n,
do ydrmane, geor ge
m
ed
bu ohwd, we , gr wi ge orge
kh
ye atsh, am mabber,eg n 3de
lts am ba ed ur an
in, i, rb , khee
bo mohara alfn drew
ris am an
n m kh
ad am
is
n
lsoan
ne lm
la, sa
de , n el l
vio
an die iva ich so
m sh dl, , m r,
sil l r
i, se
ru un tle
len ch rto on jes ael cam
ao ha sc , ch os
, ha
wa rlu on
su ks mi ya ra rd g s
be is, ho ab in s
jac la ln, ha ma
lew
ric lor
de coar, el, tho rnonb
lin az ta, rl ve
lug
ca
ma lotis, n, c rnard
gu so be , bo l j
,
lew
nienaldrio
ma rik on
or
ke ks , da
do ma
jac arl , x, mittalbert
n
pe mpieu y, s,
tru ne
vid rily ay
lem ale, da ma linds ce r tonio
romnzde e, , bru ce an
go nro ort n, en n
roh
np tee sp
mo ve gs , ch, juack k
h
bu y
da rin am
sp rah ran er, fre olla rce dy
ab ma alt jef ruh, me ne
sa ow g, i,
ken
sh llin eingham
ski omnin william a
a
ul
kh
rge
cun ith, pa lee nic geo o
sm er, a, mo os, hug
fein occ ky, iq oul rd ham
iac ins tar
rod
lew z, nop nha
g jr
azi pha berhillary
ste tz, , john ie
goe ton s, ros
clin ert ell, y
rob onn gra e
c
o'd is, mik i
nis
dav
jod
a
on,
tys , m den p ard
rell co, , lee rich iel e
vac wn dt, dan rles
bro har d, cha j
gep skin y, rick
libe sslepat ho
rk
gra hy,
l
lea wen , ma
lee, wiren, pau
tros
mcg ma eric
new n, phil ali, bou
lipto ms, −gh har al− e
sim tros bas rles
bou ad, r, chae
ass ume bruc
sch bitt,john
ry a l
bab e, r, henos sau
cag inge carl
kiss em,les
t
men n,
new b
aspi rich, , jack ld h
ging stein, donaon r
weinsfeld , nels
rum dela ica
man s, mon eth
sele , kenn
ael d
boss , barry
diller er, michmou
s
coop s, mah
jr
abba an, harry
don
trum natta,michael
van on, herman a
jacks rds, annik
edwa stam,marlin
soren ter,
fitzwa r, eliot
spitze , phil vicente
da,
gramm
e
fox quesa
georg
oud
bush, gton, george
mahm
washin inejad, a
j
ahmad , richard
grasso an, patrick
c
buchan, frank
sinatra ke, richard
de
holbroo , jon s
barrios
corzine ro, violeta
chamor richard
strauss,vince
carter, , willie
l
randolph
lay, kenneth
s
kaye, judithrichard
sandomir, deborah
solomon,
harris, eric , abby
kennedy
goodnough
onassis, jacqueline
king, stephen
erlanger, steven
ronald lyle
goldman, sandra
day
o'connor,
pol pot
cone, david
taylor, charles
dodd, christopher j
williams, jayson
najibullah
muhammad, john allen
kerkorian, kirk
weill, sanford i
roosevelt, franklin delano
tharp, twyla
lewis, neil a
childs, chris
wilson, robert
brown, ronald h
icahn, carl c
presley, elvis
agassi, andre
crew, rudolph f
johnson, philip
hevesi, alan g
chang, michael
freud, sigmund
alexander,
balanchine,lamar
bakker, jim george
zuckerman,
mortimer b
chun doo hwan
jefferson, thomas
matsui, hideki
brown, larry
kerrigan,
nancy
tierney, john
zedillo, ernesto
barak, ehud
penningt
handel, on, chad
adams, george
perez gerry frederick
botha, de cuellar,
javier
jacksonp w
quindle , thomas
rehnqu n, anna penfield
deaver ist, william
mcnall , michae h
bowe, y, terrencl k
sather riddick e
bush, , glen
hardin george
w.
putin, g, tonya
de niro,vladim
fasse
roberir
t
perel l, jim
khod man,
greenorkov ronald o
lager span sky, mikha
reid, feld, , alan
il b
boxe harry karl
wein r, barb
wilpo er, tim ara
mug n,
men abe,fred
griffi ende robert
wilsoth, z, robe
bolto n, michael rt
farro n, valerie
plam
mad w, john
e
ada onnmia r
sala ms, a
hu meh john
rafs jinta , moh
jagr anjao
amm
alle , jaro ni, has
ed
a
mir hem
stra n,
kerr wbewoo
i
she y, rry,dy
ford vardjohn darr
yl
bra , ger nad f
bus dy, ald ze,
hel h, nich rud edu
nav ms jeb olas olph ard
f
abd rati ley,
jr
col el lovaleona
eis lor rah , ma
gor enh de ma rtin
will don ow me n, om a
ar
lau iam , er, llo,
bor ren s, jeff dw fernand
cas k, , ralptennes ight dav
o
sco tro,rob
see
id
ste rse fideert h
wie inb se, l h
goo sel ren ma
mla den, ner rtin
fra dic , elie , geo
dw
zemnco , rat
rge
igh
for
,
lott resin, joh ko t
mc , terjian n
ga caltre , g
pa tes l, nt doug
kw var , h
las
fer an ottwil car
r
ma gu , i, liam l
lie iler so michlucianh
n, elle
sa be
rob fir, rm, no colin o
he
ho an rm
no rbeert wa , an
roc rto rt,son, rdjosep
eb
n,
h
cu beke gabo ma
ch om rs,r, joh le b rion i
ka av
n
g
lou lik ezo, anberna
ow ,
sh
sc osim , hu dre rd
fer hw taka, abpe go w j
ter
m
tud
le na arz ov ne
wa pejmanndez, chich r s
,
ka
lau rn n, , , josarlesdm
ve ran, er, jea franjo
ep itri
joh n−
re de
po garsacer, rodonn n ma h a
jen
ah llo n, , na a w rie
ro nin ck ed gia ld
ar seer gs , jac wa nn s
m m nbn, be
cc str au , pe ks rd v i
on on m rti te on
ne g, , e r
ll, lanya
m cenk
itc
el
h
n
ge
lto
or
sa d
, gethur l ul l
ey ar re pa chae
cs , lau
ap
ve herk, witz,f, mi vel ar
las
as lfo tof ste
cla er s, nie
pe ho
wo be da
ch i, , al shar, nicm k
for re , tta llia b
do
mo es pe wi rt
lan j
go
be
jam opith, , ro davez, orson
sc
sl
sm ich n, deharri ar
re ow an om l ma
g
br rn n, , bil tho
k
he ldi ya y, , i
rd
tric
go nadle an tik ha
mi dm r, , ric ali
l pa
3d
bra
ir
frie rbe anenei,me danie
em
,
ba rm
da am ne n, y j georg
kh hayniha gu , lisar,
ka
la,
ne rk
j
mo lel erg
ma
ve inb ren
los
liam
ste inb n,ralph
car
ste ksor, , wil el
jac de ett lion er rtari,n
na nn , r, rog go ova is saddiq
nn
be pin
jos ere s de, rad
l de ed
fed ina zicski, mm
ha
sal ad
kar low, mo ra
p r
koz eh lauh
ard nde
od h,
bus ullarich eval h
abd ls, ld,
mil yfie , pau erie
hol eill , val n e
o'n me joh jos i
pla jor, o, ardy k jr
h
ma secedwsle
jr
can h, we ard
nt is
koc rk, how
cla er, s, brery lou o
bak ple hen com
sta es, , gia rc
gat cini ma rray g
puc bs, mujohn by
jaco ss, , bob
cha land e, on vez
row ntin
per
vale r, jaysraf, jeff
blai har dy, es
mus gun jam
van on,charlie rto s
barr d,
albe am
in
war ori, s, willif
fujimsion ard benjam
ses bernhu,
law, nya h, rush
neta aug ronald
limb an, k
reag , dere, keith
jeter hornoprah
van rey, tom , robert
winf ine, orpe zo
glav pleth , alon
map rning, f w
mouklerk jason
r
de bi, g, frank ro
giam nber , ryuta
laute imoto n
hash natha
lane, s, caryn
jame ey, don is
rald
fitzge
chan an, denn
rodm dy, john
kenne lin, tom
cougher, arlenn
spect sheldo kyi, daw
silver, san suu j
aung andrew e
stein, harriet
miers, patrick j
ewing, charles, viktor s
hynes, myrdin
cherno
s jr
els, ernie
marion
barry, z, pedro
martinepaul
o'neill, , timothy james
mcveigh
er
jesus christ
s, christoph
columbu leon e
panetta,eliot l
spitzer, jesse
helms,laurent, yves
saint
edward
mcmellon,
rather, dan
klebold, dylan
charles, eleanor
risen, james
d
husband, rick
tower, john g h
souter, david
berlin, irvingg
hatch, orrin
pinochet, augusto
olmert, ehud
wright, jim
bhutto, benazir
mehta, zubin
foreman, george
wilder, l douglas
richter, mike
lazio, rick a
ferrer, fernando
marcos, ferdinand e
benedict xvi
koresh, david
kasparov, gary
kennedy, anthony
king, martin luther m
knoblauch, chuck jr
iverson, allen
trump, donald
reeves, dan
zarqawi,
simpson, abu musab al−
brown
brady, loisnicole
smith
broad, william
j
daley, richard
m
zimmer,
gehry, richard a
altman,frank
lawrence
bach, johann
k
assad,
sebastian
selig, hafez al−
hewitt,bud
dershowlleyton
pagones itz, alan
codey, , steven m
christoprichard j a
dylan, her, warren
m
vallonebob
jordan, , peter
mouss michaef
l
blumenaoui, zacaria
ryan, thal,
s
milkennolan richard
dasch , micha
sweenle, tom el r
bratto ey, john
n,
j
traub, william
mcco james
j
wells ol, william
silver , david
c
sosa stein,
weld , sammlarry a
matti , willia y
ryan ngly, m f
white, geor don
schw , maryge
conn arze jo
boes ors, negg
broz ky, jimm er, arno
ld
bin an, ivan f y
roddlade nadine
john ick, n, osam
oba son, andy a
libbyma, ben
ega , i bara
hub n, lewis ck
john belledwardjr
sha son , web m
mill rpto , keysster
wei er, n, al hawl
n
irab ner, gifford
puti u, anth
jack n, hideki ony
d
ann sonvladimir
rich an, , phil
v
d'am, frankofi
wrig ato k
clin ht, ,
ste ton franalfonse
rub rn, , che k lloy
m
kop in, how lsea
d
koh pel rob ard
rod l, hel, tedert e
nel ger
o'ro son s, mu
win urk , richt
ma fiel e, lem ard
cha xw d, andrick
mit wlaell, dav rew jr
gra che , rob e
p
be f, ll, kal ert
joh rke ste geopan
ste n , ffi
rgea
cru wa pa rich
j
giu ise rt, ul ard
l
ker lian , maii
ca rey tom rth
be rro , i, rud
a
ha eth ll, bo olp
fei wk oveseanb
hw
ari ng ins
el− sti old , yun, lud
bro ha de , rus
su wig
wil
ge ,
eli liade , jea se f k va
n
gin za msur, mawa n−bell d
dih
be ,
mi
mu ttesburgth venurtin rtra
nd
ii,
co
rra
sh heeller, nd, rut qu s
jon ay n, rob, h ee
fra ba n
by
s,
lev rd,es, chwillia ert nc de of
gre
roy ris
ba
ois
mo rkly, harob jr topm s s iii r
at
wa
er
bri
he
kim leszart,ey, rold t c
tai
r
ch o
sa me
n
a, wo ar
pin
lec lfg les
ca mp
ten iel ra lm
ha rd la, s, an h an
go et, oso, lou pe , mi g am
alb no
ch tb ve ge fer te chae ad
ho enrig aum r, doorge na
eu
l
lde shht, , nn j nd
s
o
n, ui−madbetsy a
he
ste bia ele
nr
iqu
ph n ine
en
e
k
Jonathan Chang, Jordan Boyd-Graber, David M. Blei
Connections between the Lines
Topic Models
Nubbi leverages the machinery of topic modeling (Blei et al.
2003; Hofmann 1999; Steyvers and Griths 2007), Bayesian
mixture models of discrete data.
Topic models have emerged as powerful tools for unsupervised
analysis of large document collections.
The multinomial parameters of the mixture components are
known as \topics."
Example (NYTimes using LDA)
law
lawyer
justice
judge
investigate
prosecutor
art
music
lm
artist
art
ballet
Jonathan Chang, Jordan Boyd-Graber, David M. Blei
politics
republican
democrat
senate
campaign
mayor
sports
game
coach
player
play
match
Connections between the Lines
Model input
We take as input text whose references to entities have been
identied, and convert this into a collection of documents.
1 When Jesus had spoken these words, he
went forth with his disciples over the brook
Cedron, where was a garden, into the
which he entered, and his disciples.
2 And Judas also, which betrayed him,
knew the place: for Jesus ofttimes resorted
thither with his disciples.
3 Judas then, having received a band of
men and officers from the chief priests and
Pharisees, cometh thither with lanterns and
torches and weapons.
Jonathan Chang, Jordan Boyd-Graber, David M. Blei
4 Jesus therefore, knowing all things that
should come upon him, went forth, and
said unto them, Whom seek ye?
5 They answered him, Jesus of Nazareth.
Jesus saith unto them, I am he. And Judas
also, which betrayed him, stood with them.
6 As soon then as he had said unto them, I
am he, they went backward, and fell to the
ground.
7 Then asked he them again, Whom seek
ye? And they said, Jesus of Nazareth.
Connections between the Lines
Model input
Jesus
spoken words disciples brook Cedron
garden enter disciples knowing things
seek asked seek Nazareth
1 When Jesus had spoken these words, he
went forth with his disciples over the brook
Cedron, where was a garden, into the
which he entered, and his disciples.
2 And Judas also, which betrayed him,
knew the place: for Jesus ofttimes resorted
thither with his disciples.
3 Judas then, having received a band of
men and officers from the chief priests and
Pharisees, cometh thither with lanterns and
torches and weapons.
4 Jesus therefore, knowing all things that
should come upon him, went forth, and
said unto them, Whom seek ye?
5 They answered him, Jesus of Nazareth.
Jesus saith unto them, I am he. And Judas
also, which betrayed him, stood with them.
6 As soon then as he had said unto them, I
am he, they went backward, and fell to the
ground.
7 Then asked he them again, Whom seek
ye? And they said, Jesus of Nazareth.
received band officers chief
priests Pharisees lanterns
torches weapons
Jonathan Chang, Jordan Boyd-Graber, David M. Blei
Connections between the Lines
Judas
Model input
1 When Jesus had spoken these words, he
went forth with his disciples over the brook
Cedron, where was a garden, into the
which he entered, and his disciples.
2 And Judas also, which betrayed him,
knew the place: for Jesus ofttimes resorted
thither with his disciples.
3 Judas then, having received a band of
men and officers from the chief priests and
Pharisees, cometh thither with lanterns and
torches and weapons.
4 Jesus therefore, knowing all things that
should come upon him, went forth, and
said unto them, Whom seek ye?
5 They answered him, Jesus of Nazareth.
Jesus saith unto them, I am he. And Judas
also, which betrayed him, stood with them.
6 As soon then as he had said unto them, I
am he, they went backward, and fell to the
ground.
7 Then asked he them again, Whom seek
ye? And they said, Jesus of Nazareth.
betrayed knew place disciples answered
Nazareth saith betrayed
Jonathan Chang, Jordan Boyd-Graber, David M. Blei
Connections between the Lines
Jesus
and
Judas
Intuition
Both individuals and pairs of people can be described by
topics.
Nubbi hypothesizes that each word in a pair context describes
either one of the entities or their relationship.
In 1938, Wyman co-starred with Ronald Reagan. Reagan and
actress Jane Wyman were engaged at the Chicago Theater and
married in Glendale, California. Following arguments about
Reagan's political ambitions, Wyman led for divorce in 1948.
Since Reagan is the only U.S. president to have been divorced,
Wyman is the only ex-wife of an American President.
Jonathan Chang, Jordan Boyd-Graber, David M. Blei
Connections between the Lines
Intuition
Red words can be explained by a politics topic.
Blue words can be explained by an acting topic.
politics words can be attributed to Ronald Reagan.
acting words can be attributed to either
Ronald Reagan or Jane Wyman.
The remaining words characterize their relationship.
In 1938, Wyman co-starred with Ronald Reagan. Reagan and
actress Jane Wyman were engaged at the Chicago Theater and
married in Glendale, California. Following arguments about
Reagan's political ambitions, Wyman led for divorce in 1948.
Since Reagan is the only U.S. president to have been divorced,
Wyman is the only ex-wife of an American President.
Jonathan Chang, Jordan Boyd-Graber, David M. Blei
Connections between the Lines
Graphical Model
αθ
...
Ne''
θe''
θe'
θe
z
z
z
ηθ
w
βθ
w
Ne'
w
Ne
Kθ
αψ
απ
ψe,e'
πe,e'
z
c
w
ηψ
βψ
Ne,e'
Kψ
M
N entity contexts
Kθ entity topics
M pair contexts
These intuitions are encoded in this graphical model.
Jonathan Chang, Jordan Boyd-Graber, David M. Blei
Connections between the Lines
Kψ relationship topics
Graphical Model
αθ
...
Ne''
θe''
θe'
θe
z
z
z
ηθ
w
βθ
w
Ne'
w
Ne
Kθ
αψ
απ
ψe,e'
πe,e'
z
c
w
ηψ
βψ
Ne,e'
Kψ
M
N entity contexts
Kθ entity topics
M pair contexts
Kψ relationship topics
We must then infer the posterior probability over the hidden
variables (e.g., the hidden topics and the assignments of
topics to people and relationships) given the data.
Jonathan Chang, Jordan Boyd-Graber, David M. Blei
Connections between the Lines
Graphical Model
αθ
...
Ne''
θe''
θe'
θe
z
z
z
ηθ
w
βθ
w
Ne'
w
Ne
Kθ
αψ
απ
ψe,e'
πe,e'
z
c
w
ηψ
βψ
Ne,e'
Kψ
M
N entity contexts
Kθ entity topics
M pair contexts
Kψ relationship topics
The posterior lacks the structure for ecient computation, so
exact posterior inference is intractable. We appeal instead to
mean-eld variational inference (Jordan et al. 1999).
Jonathan Chang, Jordan Boyd-Graber, David M. Blei
Connections between the Lines
Bible Topics
Relationships are described by relationship topics
(c.f. Bhattacharya et al. 2008; Newman et al. 2006; Rosen-Zvi
et al. 2004; Culotta et al. 2005; Rabbat et al. 2006).
Topics cluster words which are used to describe similar
relationships and pairs who participate in similar relationships.
People in the Bible
Top pairs
Top words
Topic 1
Topic 2
Jacob-Laban
Miriam-Moses
Ishmael-Sarah
wife
abraham
daughter
child
call
Adonizedek-Piram
Abraham-Birsha
Birsha-Lot
king
cave
lord
smote
great
Jonathan Chang, Jordan Boyd-Graber, David M. Blei
Connections between the Lines
Describing relationships
Nubbi can predict words in each entity pair's text prole.
This describes the relationship between the entities.
bible
−5.4
biological
wikipedia
●
−6.5
−5.9
●
−5.5
●
●
●
●
−6.4
−5.8
●
−7.5
−6.3
−5.7
−6.2
●
−7.0
−6.1
−5.6
−6.0
●
●
−6.5
−5.9
Word Prediction Log Likelihood
●
●
6
8
10
12
14
16
18
Number of topics
●
Nubbi
6
8
10
12
14
16
18
6
8
Number of topics
Author−Topic
Jonathan Chang, Jordan Boyd-Graber, David M. Blei
LDA
10
12
14
Number of topics
Unigram
Connections between the Lines
Mutual
16
18
Predicting entities
Nubbi can also predict which entities are best described by a text
prole.
biological
bible
wikipedia
−5.5
−4.0
●
−4.5
●
●
−4.5
●
●
−5.0
●
−5.5
−6.0
−5.0
●
−6.0
●
−7.0
−6.5
−6.5
−5.5
●
●
●
−6.0
Entity Prediction Log Likelihood
−4.0
●
6
8
10
12
14
16
18
Number of topics
●
Nubbi
6
8
10
12
14
16
18
6
8
Number of topics
Author−Topic
Jonathan Chang, Jordan Boyd-Graber, David M. Blei
LDA
10
12
14
Number of topics
Unigram
Connections between the Lines
Mutual
16
18
e ls m
ike pp ae llia y m
m se ich wi en
a,
m e, vg
zz i, giu , ar ye y v
er
pia rdkakisespeov, gu
ve ak ak i,
nif rry
du
sh im linarspikei, jen, ga s
pr , iat ov
tie
mo pr ar
ma
lee sp , al tho chris
ee
,
ca
ka ter e, an−yo tonyx sir
lei esitm
l
wi , yoner, no , de
wh
len latrel t
ma sh is, ington
ku sh ell, ren
lau ry
lew
wa rew a, , jersteve m
ny
sp bilwn s, fay
ert rvin en
ka
bro illip nt,, rob, ea antho
ph l
ph ce
i,
vin tesnson sin , stehard k
ga ma im ric rrich
joh
he
de
tomnddsky,an, mas ith i
so
ke
bro lem , tho r, , hosn
co an he
ke ds rakbarrygustglen
bra ba s, au y a
r el−
mu nd n, ne rtin
ma
bo so rodma
wil g, , b am e jr
kin gis bo muboon w
hin le, afi, t nd n
do dd s, mo phoiii
qa ken ray ste l
pic ly, ry, l paue mas
kel rbu r, kob tho
ma me , b
bre ant no, n n
bry isa joh joh dam
gol nn, n, sad
a
gle dde n,
bar h
ma sei , tom
ralp
hus kin s, bar
iel
har lter nthal, l
dan
wa me ted er ra,
blu er, oliv ved
turn th, saa, a m l
nor ga hal pau
orteent ue,
ros liabjay hak l
tag , yitz hae
lenoin, mic y
rab an, andene
poll itte, eug
pett ill, woo as r
o'ne tae thom
roh zzi, tom, nicolae
suo aw, scu ad h jr
brokuse ahm rles
cea labi, cha
cha ing, isiah
in
keat as,
k
thom r, melv
milleris, mar
o m an
mor itt, eric
schm o, mari
ge herm
cuom geori
ruth, ein gary a
huss field, imeld
shef os,
mj
marc , willia
perry karl leon
ia
rove, ein,
rd
botst , c virgin
leonaer m
fields tein, sumn
bernsone, a alfred
ouk
redst an,
taubmom sihan
lee
norod , john s s
malvo charle james e
robb, evey,
mcgre , ariel j
sharonhenry
stern, , robert
moses nd, strom
thurmo thabo
mbeki, ey, brian
mulronwayne h jr
king,
, alton
maddox n, dave
anderso todd s
purdum,
i
klein, joel
john da jr
dingell,
samuel
alito, bill
frist, bill
cosby,elizabeth h
dole,
feld, eliot
hariri, rafik
williams, serena
hart, garyalbert
einstein,
r jr
biden, joseph s
foley, thomas e
shalala, donna
manning, eli
sistani, ali al−
martinez, tino
bonilla, bobby
franks, bob
beckett, samuel
eisner, michael d
abramoff, jack
lewis, anthony
padilla, jose
winerip, michael
edwards, john
john, elton
henderson, rickey
barenboim, daniel
gonzalez, elian
moore, michael
becker, boris
vance, cyrus
pirro, jeanine r
kennedy, edward
m
holtzman,
feinstein, elizabeth
dalai lamadianne
robbins, jerome
warhol, andy
fehr, donald
pitino,
james, rick
gates, lebron
mobutubill
sese seko
ali, muhamm
ad
noriega,
rangel, manuel antonio
kristof, charles b
nicholas
levin, carl
d
wassers
thomps tein, wendy
rostenk on, william
c jr
starr, owski,
shevarkenneth dan
sullivandnadze,w
eduard
,
chirac, louis
a
abram jacque w
martin s, roberts
roth, , steve
izetbephilip
musc govic,
summhamp, alija
herbe
green ers, lawre
rt
wood , richar
nce h
joyce ward, d r
arafa , jame bob
zedil t, yasir s
gretz lo ponc
ward ky, wayne de
mille , benja
e leon, erne
florio r, judith min
sto
dean , jame
east , howas j
chre woo
rd
farra tien, d, clint
falw khan jean
man ell, , louis
geff delajerry
, winn
john en, davi
scot son,
d ie
mag
weic t, byro
mox ker, n ic
pelo ley, lowe
john si, mar ll p
difra sonnancytha jr
free nce , lynd
mus h, sco on
bru sinaloui , don bain
ald es
mill der , s j
t
comer, , thomike
sun bs, arth ma
rice unu seaur s
, joh n
lam ,
lett ontcondol n
con erm , ned eezh
kle ner an,
za
gre in, , dendav
id
pau enhcal
buf lso ousvin nis
ste fett n, e,
cle inb , wahen ste
kan me erg rre ry ven
cha tor, ns, , joe n m
bak vez microg l be jr
bru er, ,
er
bia ni, jamjuliokey
kac gg fra
ces
car zyni, nk es a ar
3d
roh luc ma
kar aty ci,ski rio
kim zai n, fra, the
jian
, feli nk od
gre da ha x c ore
sch eng ze e junmid g
j
we
, mi g
riv lchundlema n
rk
kim era , joh
r, bre
mc
,
bro enjon ma n f t
we
roeg il rianojr d
wa llstdy,
de gn jan , joh
mu lay er,one, e e n
ov
,
tay itzrdochtomrichapaul
,
mo
rd
qu lor mi ,
go ayrge , lawch rup
sc lde nth renael ert
ca ow le,
ho rte cron, hodanau, ce
rob
bli
nu x, ustonr, bil ft, wa
be
br rd er t
ed ss ha , l
en
m
re lki ba ns all
t
bir nobergn, lisum an
ro
ha bind, , jan, a , he
m nk solar etstefan dd
ry
su an
a
pir lliv tles, ton,
ro an , m jac
, jea , m
kie
an ick
nin dr ey
e ew
f
NY Times social network
a
on
em
, an x
llis , alem cia
co ez to tri ly
rto igu ard, , pa
ha dr pp an on ato
ro wm jas, an ul
sto d, ov pa l rnie j f jr
bo rp , bil be ew m
kid on n, , dr
llia io
ka
sim nto mso, an, wi rg h
cli llia
gio
wi an leyni, ep iv
y
sp ck , jos raj
nn jr
bu malin hi, n ke n f a
h
ar
sta ndg, dorson,y, joh m
ga
llia ter ret
kin de edl, wi pe rga b n
an nneil s, ma rd
ke rtin er, na nto
o'n tch leo wy
,
ma
x
tha ndrsalis,
sa lcolmphilipte ou
,
ma
ad h
ma ss vicen
am ria n
gla ,
v c iv
fox llo, , mar, joh
dia rey novaclaeph
ca on l, jos
o'c ve n, te hard
ha so pe, ric n
l
wil e,
y hael ard
ros itch , joh
rich
rav rks , vija
sta gh , mich, ri
sin son urg ter
wil rnb o, perpat ine a
tho iav jas ,
sch ns, songerald
joh ert , charlesp m
rob aro ven es
ferr ley,ste jam
oak s, s, iam ine
job ord will her
jeff ire, katjohn n
g
saf ris, n, briad
har cai an, lloy my
mc hm n, , tom
cas tse son r p
ben mp tige rgenis
tho ds, geo
woo ltz, , j den
shu tertrobert
j
has r,
pea isto williamid
chr ey, dav rt ihiro
cas et, robemor
maman, wa,
beth
altm okarudy mary
hos , ad, k
ud
crewehe , mar
ed dao
whit sier by
mes , bobdick on c d rash
holikney, , ram ame
che ines li, mohy b
cort wha harr w
al−'o sley, ge ak
helm , geor
bush ir, yitzh
dore
sham n, ilan
, theos
ramoeveltjohn
us
roos in,
mcca y, dick rd milho
arme , richa
nixonaus, jack ela
nickl tini, gabri
saba g, j k ing av
rowlin xiaop vojisl il s
deng nica, mikha
kostu chev,joe
gorba na,
montalls, bill
parce ross
iro
perot, i, junich
koizum jimmy
carter, g, curtda al−
schillinmokta
sadr, , davidj
brooksporter lynn
goss, , terry
nicholshenry j
hyde, , ted
williams t, florence
fabricanjames
levine, nancy
reagan, e, vinny
corbin
testaverd
jones, paula
henry a
waxman,
blair, tony , bill
richardson
paul e
tsongas, tawana
brawley,edward
albee,
johnson, randy
daly, john
lopez, jennifer
holland, bernard
graham, bob
pareles, jon
brown, edmund g jr
martin, curtis
volcker, paul a
maslin, janet
stern, david
levitt, arthur jr
wilson, pete
whitman, christine todd
pope
poindexter, john m
ellington, duke
courier, jim
bruno, joseph l
hun sen
snow, john w
kemp, jack f
oates,
joyce
carol
levy, steve
waldheim, kurt
strahan, michael
powell, colin l
lewinsky,
nunn, sammonica s
camby, marcus
ozawa, seiji
mcfarlane,
robert c
martin,
riley, patbilly
krugman,
dimaggio, paul
messinge joe
webber, r, ruth w
mickelso chris
savimbi, n, phil
bloombe jonas
da silva, rg, michael
li peng luiz inacio r
lula
charles
, prince
van gogh,
of wales
graham
vincent
maliki, , martha
pierce, nuri kamal
collins samue
al−
canby, , kerry l r jr
badillo vincen
papp, , hermat
prodi, joseph n
weins roman
o
lipsyt tein, harve
o'nea e, rober
y
foste l, shaqu t
pack r, vince ille
spiel wood nt w
kelle berg, , rober jr
bumi r, bill steve t w
picas ller, elisa n
pere so, pablo
beth
steves, shim
bettm ns,
on
thom an, scot
kerr as, garyt
hill, y, johnclarence
milo anita
wein sevi f
reag gart c, slob
belic an, en, oda
scio hick rona randi n
gott lino, , bill ld wils
on
hem i, johnelai
ne
mar ingw
fujimtin, ay,
mur ori, ken erne
kim phy albeyon st
aqu you , rich rto
torr ino, ng ard k
mu icel corasam
leo tom li, rob zon
lind nar bo, ert c
ack ros d, sugdike g
sca erm , eric
ar mbe
ray
ma lia, an,
ridg lon ant felic
torr e, e, oni ia
mc e, tomjohn n
c
col car joe
miy lins tne
spi aza , gley,
bro tze wa nnpaul
gor wn r, ,
bak e, , elliotkiic
gib er, albdav l hi
len son al ert id m
jr
wa no ,
sch lsh n, me
dia rod , joh l
sch na law n
jon les , er, gerence
do es, prince
rha e
so le, inger,
ss rd
as ros elizmario
art of
ma hc ,
ge su rof ge abethn hur wales
m
sim rst r, t, org
jr
sim ne ku joh e
pa ps r, rt n
ke teron on lou
we vo no, ne, o is
do inb rki , il j v jr
lee le, ergan joe
,
gia
ba tch rob er, jac
din rysmatti,, bri ert ca k
do
j sp
ba mikin hn a an
ar
vo ke s, iko ba
w
joh lpe r, ngo, davidv, rtle
mi
hit
ru
ge lernson, jus ss pla n kh tt
pa
ail
ell cid
me rsh , ad , tin
o
no tak wi olflarry a
llo es i, n,
do ydrmane, geor ge
m
ed
bu ohwd, we , gr wi ge orge
kh
ye atsh, am mabber,eg n 3de
lts am ba ed ur an
in, i, rb , khee
bo mohara alfn drew
ris am an
n m kh
ad am
is
n
lsoan
ne lm
la, sa
de , n el l
vio
an die iva ich so
m sh dl, , m r,
sil l r
i, se
ru un tle
len ch rto on jes ael cam
ao ha sc , ch os
, ha
wa rlu on
su ks mi ya ra rd g s
be is, ho ab in s
jac la ln, ha ma
lew
ric lor
de coar, el, tho rnonb
lin az ta, rl ve
lug
ca
ma lotis, n, c rnard
gu so be , bo l j
,
lew
nienaldrio
ma rik on
or
ke ks , da
do ma
jac arl , x, mittalbert
n
pe mpieu y, s,
tru ne
vid rily ay
lem ale, da ma linds ce r tonio
romnzde e, , bru ce an
go nro ort n, en n
roh
np tee sp
mo ve gs , ch, juack k
h
bu y
da rin am
sp rah ran er, fre olla rce dy
ab ma alt jef ruh, me ne
sa ow g, i,
ken
sh llin eingham
ski omnin william a
a
ul
kh
rge
cun ith, pa lee nic geo o
sm er, a, mo os, hug
fein occ ky, iq oul rd ham
iac ins tar
rod
lew z, nop nha
g jr
azi pha berhillary
ste tz, , john ie
goe ton s, ros
clin ert ell, y
rob onn gra e
c
o'd is, mik i
nis
dav
jod
a
on,
tys , m den p ard
rell co, , lee rich iel e
vac wn dt, dan rles
bro har d, cha j
gep skin y, rick
libe sslepat ho
rk
gra hy,
l
lea wen , ma
lee, wiren, pau
tros
mcg ma eric
new n, phil ali, bou
lipto ms, −gh har al− e
sim tros bas rles
bou ad, r, chae
ass ume bruc
sch bitt,john
ry a l
bab e, r, henos sau
cag inge carl
kiss em,les
t
men n,
new b
aspi rich, , jack ld h
ging stein, donaon r
weinsfeld , nels
rum dela ica
man s, mon eth
sele , kenn
ael d
boss , barry
diller er, michmou
s
coop s, mah
jr
abba an, harry
don
trum natta,michael
van on, herman a
jacks rds, annik
edwa stam,marlin
soren ter,
fitzwa r, eliot
spitze , phil vicente
da,
gramm
e
fox quesa
georg
oud
bush, gton, george
mahm
washin inejad, a
j
ahmad , richard
grasso an, patrick
c
buchan, frank
sinatra ke, richard
de
holbroo , jon s
barrios
corzine ro, violeta
chamor richard
strauss,vince
carter, , willie
l
randolph
lay, kenneth
s
kaye, judithrichard
sandomir, deborah
solomon,
harris, eric , abby
kennedy
goodnough
onassis, jacqueline
king, stephen
erlanger, steven
ronald lyle
goldman, sandra
day
o'connor,
pol pot
cone, david
taylor, charles
dodd, christopher j
williams, jayson
najibullah
muhammad, john allen
kerkorian, kirk
weill, sanford i
roosevelt, franklin delano
tharp, twyla
lewis, neil a
childs, chris
wilson, robert
brown, ronald h
icahn, carl c
presley, elvis
agassi, andre
crew, rudolph f
johnson, philip
hevesi, alan g
chang, michael
freud, sigmund
alexander,
balanchine,lamar
bakker, jim george
zuckerman,
mortimer b
chun doo hwan
jefferson, thomas
matsui, hideki
brown, larry
kerrigan,
nancy
tierney, john
zedillo, ernesto
barak, ehud
penningt
handel, on, chad
adams, george
perez gerry frederick
botha, de cuellar,
javier
jacksonp w
quindle , thomas
rehnqu n, anna penfield
deaver ist, william
mcnall , michae h
bowe, y, terrencl k
sather riddick e
bush, , glen
hardin george
w.
putin, g, tonya
de niro,vladim
fasse
roberir
t
perel l, jim
khod man,
greenorkov ronald o
lager span sky, mikha
reid, feld, , alan
il b
boxe harry karl
wein r, barb
wilpo er, tim ara
mug n,
men abe,fred
griffi ende robert
wilsoth, z, robe
bolto n, michael rt
farro n, valerie
plam
mad w, john
e
ada onnmia r
sala ms, a
hu meh john
rafs jinta , moh
jagr anjao
amm
alle , jaro ni, has
ed
a
mir hem
stra n,
kerr wbewoo
i
she y, rry,dy
ford vardjohn darr
yl
bra , ger nad f
bus dy, ald ze,
hel h, nich rud edu
nav ms jeb olas olph ard
f
abd rati ley,
jr
col el lovaleona
eis lor rah , ma
gor enh de ma rtin
will don ow me n, om a
ar
lau iam , er, llo,
bor ren s, jeff dw fernand
cas k, , ralptennes ight dav
o
sco tro,rob
see
id
ste rse fideert h
wie inb se, l h
goo sel ren ma
mla den, ner rtin
fra dic , elie , geo
dw
zemnco , rat
rge
igh
for
,
lott resin, joh ko t
mc , terjian n
ga caltre , g
pa tes l, nt doug
kw var , h
las
fer an ottwil car
r
ma gu , i, liam l
lie iler so michlucianh
n, elle
sa be
rob fir, rm, no colin o
he
ho an rm
no rbeert wa , an
roc rto rt,son, rdjosep
eb
n,
h
cu beke gabo ma
ch om rs,r, joh le b rion i
ka av
n
g
lou lik ezo, anberna
ow ,
sh
sc osim , hu dre rd
fer hw taka, abpe go w j
ter
m
tud
le na arz ov ne
wa pejmanndez, chich r s
,
ka
lau rn n, , , josarlesdm
ve ran, er, jea franjo
ep itri
joh n−
re de
po garsacer, rodonn n ma h a
jen
ah llo n, , na a w rie
ro nin ck ed gia ld
ar seer gs , jac wa nn s
m m nbn, be
cc str au , pe ks rd v i
on on m rti te on
ne g, , e r
ll, lanya
m cenk
itc
el
h
n
ge
lto
or
sa d
, gethur l ul l
ey ar re pa chae
cs , lau
ap
ve herk, witz,f, mi vel ar
las
as lfo tof ste
cla er s, nie
pe ho
wo be da
ch i, , al shar, nicm k
for re , tta llia b
do
mo es pe wi rt
lan j
go
be
jam opith, , ro davez, orson
sc
sl
sm ich n, deharri ar
re ow an om l ma
g
br rn n, , bil tho
k
he ldi ya y, , i
rd
tric
go nadle an tik ha
mi dm r, , ric ali
l pa
3d
bra
ir
frie rbe anenei,me danie
em
,
ba rm
da am ne n, y j georg
kh hayniha gu , lisar,
ka
la,
ne rk
j
mo lel erg
ma
ve inb ren
los
liam
ste inb n,ralph
car
ste ksor, , wil el
jac de ett lion er rtari,n
na nn , r, rog go ova is saddiq
nn
be pin
jos ere s de, rad
l de ed
fed ina zicski, mm
ha
sal ad
kar low, mo ra
p r
koz eh lauh
ard nde
od h,
bus ullarich eval h
abd ls, ld,
mil yfie , pau erie
hol eill , val n e
o'n me joh jos i
pla jor, o, ardy k jr
h
ma secedwsle
jr
can h, we ard
nt is
koc rk, how
cla er, s, brery lou o
bak ple hen com
sta es, , gia rc
gat cini ma rray g
puc bs, mujohn by
jaco ss, , bob
cha land e, on vez
row ntin
per
vale r, jaysraf, jeff
blai har dy, es
mus gun jam
van on,charlie rto s
barr d,
albe am
in
war ori, s, willif
fujimsion ard benjam
ses bernhu,
law, nya h, rush
neta aug ronald
limb an, k
reag , dere, keith
jeter hornoprah
van rey, tom , robert
winf ine, orpe zo
glav pleth , alon
map rning, f w
mouklerk jason
r
de bi, g, frank ro
giam nber , ryuta
laute imoto n
hash natha
lane, s, caryn
jame ey, don is
rald
fitzge
chan an, denn
rodm dy, john
kenne lin, tom
cougher, arlenn
spect sheldo kyi, daw
silver, san suu j
aung andrew e
stein, harriet
miers, patrick j
ewing, charles, viktor s
hynes, myrdin
cherno
s jr
els, ernie
marion
barry, z, pedro
martinepaul
o'neill, , timothy james
mcveigh
er
jesus christ
s, christoph
columbu leon e
panetta,eliot l
spitzer, jesse
helms,laurent, yves
saint
edward
mcmellon,
rather, dan
klebold, dylan
charles, eleanor
risen, james
d
husband, rick
tower, john g h
souter, david
berlin, irvingg
hatch, orrin
pinochet, augusto
olmert, ehud
wright, jim
bhutto, benazir
mehta, zubin
foreman, george
wilder, l douglas
richter, mike
lazio, rick a
ferrer, fernando
marcos, ferdinand e
benedict xvi
koresh, david
kasparov, gary
kennedy, anthony
king, martin luther m
knoblauch, chuck jr
iverson, allen
trump, donald
reeves, dan
zarqawi,
simpson, abu musab al−
brown
brady, loisnicole
smith
broad, william
j
daley, richard
m
zimmer,
gehry, richard a
altman,frank
lawrence
bach, johann
k
assad,
sebastian
selig, hafez al−
hewitt,bud
dershowlleyton
pagones itz, alan
codey, , steven m
christoprichard j a
dylan, her, warren
m
vallonebob
jordan, , peter
mouss michaef
l
blumenaoui, zacaria
ryan, thal,
s
milkennolan richard
dasch , micha
sweenle, tom el r
bratto ey, john
n,
j
traub, william
mcco james
j
wells ol, william
silver , david
c
sosa stein,
weld , sammlarry a
matti , willia y
ryan ngly, m f
white, geor don
schw , maryge
conn arze jo
boes ors, negg
broz ky, jimm er, arno
ld
bin an, ivan f y
roddlade nadine
john ick, n, osam
oba son, andy a
libbyma, ben
ega , i bara
hub n, lewis ck
john belledwardjr
sha son , web m
mill rpto , keysster
wei er, n, al hawl
n
irab ner, gifford
puti u, anth
jack n, hideki ony
d
ann sonvladimir
rich an, , phil
v
d'am, frankofi
wrig ato k
clin ht, ,
ste ton franalfonse
rub rn, , che k lloy
m
kop in, how lsea
d
koh pel rob ard
rod l, hel, tedert e
nel ger
o'ro son s, mu
win urk , richt
ma fiel e, lem ard
cha xw d, andrick
mit wlaell, dav rew jr
gra che , rob e
p
be f, ll, kal ert
joh rke ste geopan
ste n , ffi
rgea
cru wa pa rich
j
giu ise rt, ul ard
l
ker lian , maii
ca rey tom rth
be rro , i, rud
a
ha eth ll, bo olp
fei wk oveseanb
hw
ari ng ins
el− sti old , yun, lud
bro ha de , rus
su wig
wil
ge ,
eli liade , jea se f k va
n
gin za msur, mawa n−bell d
dih
be ,
mi
mu ttesburgth venurtin rtra
nd
ii,
co
rra
sh heeller, nd, rut qu s
jon ay n, rob, h ee
fra ba n
by
s,
lev rd,es, chwillia ert nc de of
gre
roy ris
ba
ois
mo rkly, harob jr topm s s iii r
at
wa
er
bri
he
kim leszart,ey, rold t c
tai
r
ch o
sa me
n
a, wo ar
pin
lec lfg les
ca mp
ten iel ra lm
ha rd la, s, an h an
go et, oso, lou pe , mi g am
alb no
ch tb ve ge fer te chae ad
ho enrig aum r, doorge na
eu
l
lde shht, , nn j nd
s
o
n, ui−madbetsy a
he
ste bia ele
nr
iqu
ph n ine
en
e
k
Jonathan Chang, Jordan Boyd-Graber, David M. Blei
Connections between the Lines
NY Times social network
trial
match
Jonathan Chang, Jordan Boyd-Graber, David M. Blei
g
tower, johnvid
h
souter, da
ing
irv
berlin,
hatch, orrin g usto
pinochet, aug
olmert, ehud
wright, jim
bhutto, benazir
mehta, zubin
foreman, george
wilder, l douglas
richter, mike
lazio, rick a
ferrer, fernando
marcos, ferdinand e
benedict xvi
koresh, david
kasparov, gary
kennedy, anthony
king, martin luther m
jr
knoblauch, chu
iverson, allen ck
trump, dona
reeves, da ld
zarqawi, abnu m
simpson, nico usab al!
le brown
brady, lois
broad, williasmith
m
daley, ha j
rd m
zimmeric
r, richard
gehry, fr
a
a
n
k
altma
bach, njo, hlawrence k
assad, h ann sebastia
n
afez al!
selig, bfight
hewitt, llud
e
y
ton
ders
pagohowitz, alan
codeyn, es, steven am
chris richard j
pher, w the Lines
dylanto
Connections
arre
, b between
8
Examining predictions on solicited descriptions
5
4
3
2
1
Log Rank (Nubbi)
6
7
fed users
prize
prize
spot
republic
married
figures
figures
married
figures
headed
previously
figures
2004
2004
headed
figures
figures
2001
married
married
iran
opponents
married
powers
september
married
politicians
opponents
married
married
politicians
brothers
politicians
victim
politicians
track stars
married
opponents
politicians
politicians
brothers
ran
politicians
ran
ran
politicians
opponents
2001
married
ran
area
eastern
victimran
married
person
candidates
subject
planned
opponents
daughter
ran
politicians 1992
sex
1996
subject
daughter
person
person
victim
clinton
married
terrorist fed
british
politicians
atlanta
1992 involved
1988
person
politicians
convicted
son
2000
politicians
professional
failed
politicians
politicians
worked
olympic
senators
convicted
professional
person
africa
person
politicians
current
politicians
politicians
bill california
senators
reagan
countries
speaker
running
owner
special
opponents
process
convicted
process
race2000
involved
senators
professional
senators
mother
groups
running
running
father
building father
bush
secretary
accused
candidates
primary
countries
candidates
candidates
talk
commissioner
owner
countries
attackbombing
running
countries
candidates
russian
candidates
candidates
countries
olympic
candidates
countries
countries
won
bush
running
running
staff
late
bush
bush
clinton
figure
middle
bush
middle
accused
involved
countries
candidates
south
east
countries
candidates
middle
east
middle
issue
middle
middle
countries
father
supreme east
countries
candidates
basketball
east
troops east
father
field
washington
supreme
russia
baseball
israeli
race
supreme
israeliisraeli
wife
part
election
republicans
candidates
tennis
israeli
democratic
candidates
part
palestinian
election leaders
vice vice
jersey
jersey
son
fatherfather
son
chief
palestinian
defense
sex
presidential
peace
peace
baseball
presidential
american
executives
vice
nations
attorney
peacepresidential
brother
attorney
presidentialexecutives
world
attorney
sex
leaders
nations
disney
presidential
husband
leaders
world
attorney
leaders
world
world
world
presidential
world
leaders
leaders
leaders
vice
leaders
leaders
leaders
prime party
world
husband
leaders
leaders
leaders
party
country
leaders
chairman
managerparty
police night
wife
leaders
political
baseball
american
baseball tennis
partyvice
investigation
baseball
democratic
secretary
secretary
players
republicans time
senate
democratic
democratic
timegeneral
democratic
democratic
vice
husband
general
players
democraticrepublicans
husband
american
iraq
time
tennis
tennis
political
political
war players
minister
show time
time
players
political
state
timegeneral
husband
republican
political state
husband
state
time time
american
husband
wife
players
police
husband
husband
wife
players
iraq
wife
wife
wife
governor
players
york
players
war
sexual
political
political
political
states republican
governor
yankees
york
relationship
americanrepublican
white
united
governor
york
republican
republican
case
court court
sexual
sexual
federal
york
case administration state
mayor administration
court
administration
city
house
president
president president
statepresident
state president
1
2
3
4
5
6
Log Rank (Author−Topic Model)
Lower is better
Jonathan Chang, Jordan Boyd-Graber, David M. Blei
Connections between the Lines
7
8
4
3
2
2001
iran
brothers
politicians
track
stars
brothers
candidates
married
fed
terrorist
olympic
africa
owner
opponents
candidates
primary
candidates
running
russian
figure
south
supreme
russia
israeli
baseball
israeli
wife
israeli
israeli
palestinian
father
son
father
palestinian
sex son
presidential
peace
peace
baseball
peace
nations husband
presidential
presidential
brother
sex
disney
presidential
nations
world
world
party
prime
husband
chairman
tennis wife
baseball
senate
democratic
democratic
democratic
husband
husband
tennis
tennis
minister iraq
time
husbandhusband
husband
wife
husband
wife
wife
wife
governor
sexual
york
case
relationship
sexual
sexual
court
administration
president
1
Log Rank (Nubbi)
5
6
7
Examining predictions on solicited descriptions
president
president
state
1
2
3
4
5
6
Log Rank (Author−Topic Model)
Lower is better
Jonathan Chang, Jordan Boyd-Graber, David M. Blei
Connections between the Lines
7
Conclusion
Networks of entities are a pervasive form of data.
There are deep meanings associated with the links in these
networks.
These meanings are often not encoded in the network itself,
but rather as free text elsewhere.
The Nubbi model allows us to take this text and infer
characterizations of the relationships between the ensemble of
entities.
This is an important step towards mining complete social
networks from free text.
Thanks!
Jonathan Chang, Jordan Boyd-Graber, David M. Blei
Connections between the Lines
Wikipedia Topics
Nubbi learns topics associated with individual entities.
This clusters entities according to the words used to describe
them.
People in Wikipedia
Top Entities
Top Terms
Topic 1
Topic 2
Frederick Sanger
Svante Arrhenius
William Ramsay
work
universe
year
develop
society
Kate Winslet
Ringo Starr
Al Pacino
lm
music
album
award
song
Jonathan Chang, Jordan Boyd-Graber, David M. Blei
Connections between the Lines
Wikipedia Topics
Nubbi also learns topics associated with pairs of entities.
These topics cluster together pairs with similar relationships.
People in Wikipedia
Top Pairs
Top Terms
Topic 1
Topic 2
Tyler-Roosevelt
McKinley-Roosevelt
Taft-Roosevelt
president
vice
roosevelt
calvin
johnson
Perot-Bush
J.Q. Adams-Monroe
J.Q. Adams-Clay
republican
march
reagan
vote
state
Jonathan Chang, Jordan Boyd-Graber, David M. Blei
Connections between the Lines
Graphical Model
αθ
...
Ne''
θe''
θe'
θe
z
z
z
ηθ
w
βθ
w
Ne'
w
Ne
Kθ
αψ
απ
ψe,e'
πe,e'
z
c
w
ηψ
βψ
Ne,e'
Kψ
M
N entity contexts
1
Kθ entity topics
M pair contexts
For each entity e,
1
2
Draw entity topic proportions e Dir( );
For each word associated with this entity's context,
1 Draw topic assignment ze ;n Mult(e );
2 Draw word we ;n Mult(ze ;n ).
Jonathan Chang, Jordan Boyd-Graber, David M. Blei
Connections between the Lines
Kψ relationship topics
Graphical Model
αθ
...
Ne''
θe''
θe'
θe
z
z
z
ηθ
w
βθ
w
Ne'
w
Ne
Kθ
αψ
απ
ψe,e'
πe,e'
z
c
w
ηψ
βψ
Ne,e'
Kψ
M
N entity contexts
1
Kθ entity topics
M pair contexts
For each pair of entities e ; e 0 ,
1
2
3
Kψ relationship topics
Draw relationship topic proportions e ;e 0 Dir( );
Draw selector proportions e ;e 0 Dir( );
For each word associated with this entity pair's context,
1 Draw selector ce ;e 0 ;n Mult(e ;e 0 );
Jonathan Chang, Jordan Boyd-Graber, David M. Blei
Connections between the Lines
Graphical Model
αθ
...
Ne''
θe''
θe'
θe
z
z
z
ηθ
w
βθ
w
Ne'
w
Ne
Kθ
αψ
απ
ψe,e'
πe,e'
z
c
w
ηψ
βψ
Ne,e'
Kψ
M
N entity contexts
1
Kθ entity topics
M pair contexts
If ce ;e ;n = 1,
0
1
2
Draw topic assignment ze ;e 0 ;n Mult(e );
Draw word we ;e 0 ;n Mult(ze ;e 0 ;n ):
Jonathan Chang, Jordan Boyd-Graber, David M. Blei
Connections between the Lines
Kψ relationship topics
Graphical Model
αθ
...
Ne''
θe''
θe'
θe
z
z
z
ηθ
w
βθ
w
Ne'
w
Ne
Kθ
αψ
απ
ψe,e'
πe,e'
z
c
w
ηψ
βψ
Ne,e'
Kψ
M
N entity contexts
1
Kθ entity topics
M pair contexts
If ce ;e ;n = 2,
0
1
2
Draw topic assignment ze ;e 0 ;n Mult(e 0 );
Draw word we ;e 0 ;n Mult(ze ;e 0 ;n ):
Jonathan Chang, Jordan Boyd-Graber, David M. Blei
Connections between the Lines
Kψ relationship topics
Graphical Model
αθ
...
Ne''
θe''
θe'
θe
z
z
z
ηθ
w
βθ
w
Ne'
w
Ne
Kθ
αψ
απ
ψe,e'
πe,e'
z
c
w
ηψ
βψ
Ne,e'
Kψ
M
N entity contexts
1
Kθ entity topics
M pair contexts
If ce ;e ;n = 3,
0
1
2
Draw topic assignment ze ;e 0 ;n Mult(
Draw word we ;e 0 ;n Mult(ze ;e 0 ;n ):
Jonathan Chang, Jordan Boyd-Graber, David M. Blei
e ;e 0
);
Connections between the Lines
Kψ relationship topics
References
Eugene Agichtein and Luis Gravano. Querying text databases for
ecient information extraction. Data Engineering, International
Conference on, 0:113, 2003. ISSN 1063-6382. doi: http:
//doi.ieeecomputersociety.org/10.1109/ICDE.2003.1260786.
Michele Banko, Michael J. Cafarella, Stephen Soderland, Matthew
Broadhead, and Oren Etzioni. Open information extraction from
the web. In IJCAI 2007, 2007. URL http:
//www.ijcai.org/papers07/Papers/IJCAI07-429.pdf.
Indrajit Bhattacharya, Shantanu Godbole, and Sachindra Joshi.
Structured entity identication and document categorization:
Two tasks with one joint model. KDD 2008, 2008. URL http:
//portal.acm.org/ft gateway.cfm?id=1401899&type=
pdf&coll=ACM&dl=ACM&CFID=2420023&CFTOKEN=56734162.
D Blei, A Ng, and M Jordan. Latent Dirichlet allocation. Journal
of Machine Learning Research, 2003. URL
http://www.mitpressjournals.org/doi/abs/10.1162/
jmlr.2003.3.4-5.993.
Jonathan Chang, Jordan Boyd-Graber, David M. Blei
Connections between the Lines