To demonstrate the method we will turn two small corpi into semantic networks and compare them.
These are frivolous examples but interestin in that they have an identical set of words, but different punctuation. We will create networks between all words that occur within sentences.
Dear John:
I want a man who knows what love is all about.
You are generous, kind, thoughtful.
People who are not like you admit to being useless and inferior.
You have ruined me for other men.
I yearn for you.
I have no feelings whatsoever when we’re apart.
I can be forever happy.
Will you let me be yours?
Gloria
Dear John:
I want a man who knows what love is.
All about you are generous, kind, thoughtful people, who are not like you.
Admit to being useless and inferior.
You have ruined me.
For other men, I yearn.
For you, I have no feelings whatsoever.
When we’re apart, I can be forever happy.
Will you let me be?
Yours,
Gloria
# setwd( "C:/Users/jdlecy/Dropbox/02 - CLASSES/02 - MASTERS/03 - R Crash Course/03 - Labs/Lab 10 - Text as Data" )
# x <- readLines( "assets/dear_john_letter_1.txt", warn=FALSE )
x <- readLines( "https://raw.githubusercontent.com/Nonprofit-Open-Data-Collective/machine_learning_mission_codes/master/docs/tutorials/assets/dear_john_letter_1.txt", warn=FALSE )
x <- toupper( x )
x <- gsub( ",", "", x )
x <- gsub( "WE’RE", "WE ARE", x )
x <- gsub( "\\:", "", x )
x <- gsub( "\\?", "", x )
x## [1] "DEAR JOHN"
## [2] ""
## [3] "I WANT A MAN WHO KNOWS WHAT LOVE IS ALL ABOUT. YOU ARE GENEROUS KIND THOUGHTFUL. PEOPLE WHO ARE NOT LIKE YOU ADMIT TO BEING USELESS AND INFERIOR. YOU HAVE RUINED ME FOR OTHER MEN. I YEARN FOR YOU. I HAVE NO FEELINGS WHATSOEVER WHEN WE ARE APART. I CAN BE FOREVER HAPPY."
## [4] ""
## [5] "WILL YOU LET ME BE YOURS"
## [6] ""
## [7] "GLORIA"## [1] "DEAR JOHN"
## [2] "I WANT A MAN WHO KNOWS WHAT LOVE IS ALL ABOUT. YOU ARE GENEROUS KIND THOUGHTFUL. PEOPLE WHO ARE NOT LIKE YOU ADMIT TO BEING USELESS AND INFERIOR. YOU HAVE RUINED ME FOR OTHER MEN. I YEARN FOR YOU. I HAVE NO FEELINGS WHATSOEVER WHEN WE ARE APART. I CAN BE FOREVER HAPPY."
## [3] "WILL YOU LET ME BE YOURS"
## [4] "GLORIA"Break paragraphs into sentences:
## [[1]]
## [1] "DEAR JOHN"
##
## [[2]]
## [1] "I WANT A MAN WHO KNOWS WHAT LOVE IS ALL ABOUT"
## [2] " YOU ARE GENEROUS KIND THOUGHTFUL"
## [3] " PEOPLE WHO ARE NOT LIKE YOU ADMIT TO BEING USELESS AND INFERIOR"
## [4] " YOU HAVE RUINED ME FOR OTHER MEN"
## [5] " I YEARN FOR YOU"
## [6] " I HAVE NO FEELINGS WHATSOEVER WHEN WE ARE APART"
## [7] " I CAN BE FOREVER HAPPY"
##
## [[3]]
## [1] "WILL YOU LET ME BE YOURS"
##
## [[4]]
## [1] "GLORIA"Turn sentences into networks.
First turn sentences into tokens:
## [1] "YOU" "ARE" "GENEROUS" "KIND" "THOUGHTFUL"Then turn tokens into word pairs:
## [,1] [,2]
## [1,] "YOU" "ARE"
## [2,] "YOU" "GENEROUS"
## [3,] "YOU" "KIND"
## [4,] "YOU" "THOUGHTFUL"
## [5,] "ARE" "GENEROUS"
## [6,] "ARE" "KIND"
## [7,] "ARE" "THOUGHTFUL"
## [8,] "GENEROUS" "KIND"
## [9,] "GENEROUS" "THOUGHTFUL"
## [10,] "KIND" "THOUGHTFUL"Now apply this to all sentences, one at a time:
textToNet <- function( x )
{
x <- unlist( x )
word.pairs <- NULL
for( i in 1:length(x) )
{
x.sub <- x[i]
x.split <- strsplit( x.sub, split=" " )[[1]]
x.split <- x.split[ - grep( "^$", x.split ) ]
if( length(x.split) > 1 ) { word.pairs <- rbind( word.pairs, t( combn( x.split, 2) ) ) }
}
return( word.pairs )
}
textToNet( sentences ) %>% head()## [,1] [,2]
## [1,] "YOU" "ARE"
## [2,] "YOU" "GENEROUS"
## [3,] "YOU" "KIND"
## [4,] "YOU" "THOUGHTFUL"
## [5,] "ARE" "GENEROUS"
## [6,] "ARE" "KIND"# library( igraph )
mat1 <- textToNet( sentences )
g1 <- graph.edgelist( mat1, directed=FALSE )
par( mar=c(0,0,3,0) )
plot( g1,
layout = layout.kamada.kawai(g1),
edge.arrow.size =0.4,
vertex.size = 10,
vertex.label.cex = 1.5,
vertex.frame.color = "gray",
vertex.color = "gray"
)
title( main="Dear John Version 1" )