The series so far:<\/strong><\/p>\n This is the third article of the \u201cText Mining and Sentiment Analysis\u201d Series. The first article introduced Azure Cognitive Services and demonstrated the setup and use of Text Analytics APIs for extracting key Phrases & Sentiment Scores from text data. The second article demonstrated Power BI visualizations for analyzing Key Phrases & Sentiment Scores and interpreting them to gain insights. This article explores R for text mining and sentiment analysis. I will demonstrate several common text analytics techniques and visualizations in R.<\/p>\n\n\n\n Note: This article assumes basic familiarity with R and RStudio. Please jump to the References section for more information on installing R and RStudio. The Demo data raw text file and R script are available for download from my GitHub repository; please find the link in the References section.<\/p>\n\n\n\n R is a language and environment for statistical computing and graphics. It provides a wide variety of statistical and graphical techniques and is highly extensible. R is available as free software. It\u2019s easy to learn<\/p>\n\n\n\n and use and can produce well designed publication-quality plots. For the demos in this article, I am using R version 3.5.3 (2019-03-11), RStudio Version 1.1.456<\/p>\n\n\n\n The input file for this article has only one column, the \u201cRaw text\u201d of survey responses and is a text file.<\/p>\n\n\n\n A sample of the first few rows are shown in Notepad++ (showing all characters) in Figure 1.<\/p>\n\n\n\n <\/p>\n\n\n\n Figure 1. Sample of the input text file<\/strong><\/p>\n\n\n\n The demo R script and demo input text file are available on my GitHub repo (please find the link in the References section).<\/p>\n\n\n\n R has a rich set of packages for Natural Language Processing (NLP) and generating plots. The foundational steps involve loading the text file into an R Corpus, then cleaning and stemming the data before performing analysis. I will demonstrate these steps and analysis like Word Frequency, Word Cloud, Word Association, Sentiment Scores and Emotion Classification using various plots and charts.<\/p>\n\n\n\n The following packages are used in the examples in this article:<\/p>\n\n\n Open RStudio and create a new R Script. Use the following code to install and load these packages.<\/p>\n\n\n\n The R base function The input file has multiple lines of text and no columns\/fields (data is not tabular), so you will use the In your R script, add the following code to load the data into a corpus.<\/p>\n\n\n\n Upon running this, you will be prompted to select the input file. Navigate to your file and click Open <\/em>as shown in Figure 2.<\/p>\n\n\n\n <\/p>\n\n\n\n Figure 2. Select input file<\/strong><\/p>\n\n\n\n Cleaning the text data starts with making transformations like removing special characters from the text. This is done using the Then remove the stopwords<\/em>. They are the most commonly occurring words in a language and have very little value in terms of gaining useful information. They should be removed before performing further analysis. Examples of stopwords in English are \u201cthe, is, at, on\u201d<\/em>. There is no single universal list of stop words used by all NLP tools. The last step is text stemming. It is the process of reducing the word to its root form. The stemming process simplifies the word to its common origin. For example, the stemming process reduces the words \u201cfishing\u201d, \u201cfished\u201d and \u201cfisher\u201d to its stem \u201cfish\u201d. Please note stemming uses the SnowballC<\/em> package. (You may want to skip the text stemming step if your users indicate a preference to see the original \u201cunstemmed\u201d words in the word cloud plot)<\/p>\n\n\n\n In your R script, add the following code to transform and run to clean-up the text data.<\/p>\n\n\n\n After cleaning the text data, the next step is to count the occurrence of each word, to identify popular or trending topics. Using the function In your R script, add the following code and run it to see the top 5 most frequently found words in your text.<\/p>\n\n\n\n The following table of word frequency is the expected output of the <\/p>\n\n\n\n Plotting the top 5 most frequent words using a bar chart is a good basic way to visualize this word frequent data. In your R script, add the following code and run it to generate a bar chart, which will display in the Plots<\/em> sections of RStudio.<\/p>\n\n\n\n The plot can be seen in Figure 3.<\/p>\n\n\n\n <\/p>\n\n\n\n Figure 3. Bar chart of the top 5 most frequent words<\/strong><\/p>\n\n\n\n One could interpret the following from this bar chart:<\/p>\n\n\n A word cloud is one of the most popular ways to visualize and analyze qualitative data. It\u2019s an image composed of keywords found within a body of text, where the size of each word indicates its frequency in that body of text. Use the word frequency data frame (table) created previously to generate the word cloud. In your R script, add the following code and run it to generate the word cloud and display it in the Plots<\/em> section of RStudio.<\/p>\n\n\n\n Below is a brief description of the arguments used in the word cloud function;<\/p>\n\n\n You can see the resulting word cloud in Figure 4.<\/p>\n\n\n\n <\/p>\n\n\n\n Figure 4. Word cloud plot<\/strong><\/p>\n\n\n\n The word cloud shows additional words that occur frequently and could be of interest for further analysis. Words like \u201cneed\u201d, \u201csupport\u201d, \u201cissu\u201d (root for \u201cissue(s)\u201d, etc. could provide more context around the most frequently occurring words and help to gain a better understanding of the main themes.<\/p>\n\n\n\n Correlation is a statistical technique that can demonstrate whether, and how strongly, pairs of variables are related. This technique can be used effectively to analyze which words occur most often in association with the most frequently occurring words in the survey responses, which helps to see the context around these words<\/p>\n\n\n\n In your R script, add the following code and run it.<\/p>\n\n\n\n You should see the results as shown in Figure 5.<\/p>\n\n\n\n <\/p>\n\n\n\n Figure 5. Word association analysis for the top three most frequent terms<\/strong><\/p>\n\n\n\n This script shows which words are most frequently associated with the top three terms ( You can modify the above script to find terms associated with words that occur at least 50 times or more, instead of having to hard code the terms in your script.<\/p>\n\n\n\n <\/p>\n\n\n\n Figure 6: Word association output for terms occurring at least 50 times <\/strong><\/p>\n\n\n\n Sentiments can be classified as positive, neutral or negative. They can also be represented on a numeric scale, to better express the degree of positive or negative strength of the sentiment contained in a body of text.<\/p>\n\n\n\n This example uses the Syuzhet package for generating sentiment scores, which has four sentiment dictionaries and offers a method for accessing the sentiment extraction tool developed in the NLP group at Stanford. The Add the following code to the R script and run it.<\/p>\n\n\n\n Your results should look similar to Figure 7.<\/p>\n\n\n\n <\/p>\n\n\n\n Figure 7. Syuzhet vector<\/strong><\/p>\n\n\n\n An inspection of the Syuzhet vector shows the first element has the value of 2.60<\/em>. It means the sum of the sentiment scores of all meaningful words in the first response(line) in the text file, adds up to 2.60. The scale for sentiment scores using the Next, run the same analysis for the remaining two methods and inspect their respective vectors. Add the following code to the R script and run it.<\/p>\n\n\n\n Your results should resemble Figure 8.<\/p>\n\n\n\n <\/p>\n\n\n\n Figure 8. bing and afinn vectors<\/strong><\/p>\n\n\n\n Please note the scale of sentiment scores generated by:<\/p>\n\n\n The summary statistics of Because these different methods use different scales, it\u2019s better to convert their output to a common scale before comparing them. This basic scale conversion can be done easily using R\u2019s built-in Add the following code to your R script and run it.<\/p>\n\n\n\n Figure 9 shows the results.<\/p>\n\n\n\n <\/p>\n\n\n\n Figure 9. Normalize scale and compare three vectors<\/strong><\/p>\n\n\n\n Note the first element of each row (vector) is 1<\/em>, indicating that all three methods have calculated a positive sentiment score, for the first response (line) in the text.<\/p>\n\n\n\n Emotion classification is built on the NRC Word-Emotion Association Lexicon (aka EmoLex). The definition of \u201cNRC Emotion Lexicon\u201d, sourced from http:\/\/saifmohammad.com\/WebPages\/NRC-Emotion-Lexicon.htm<\/a> is \u201cThe NRC Emotion Lexicon is a list of English words and their associations with eight basic emotions (anger, fear, anticipation, trust, surprise, sadness, joy, and disgust) and two sentiments (negative and positive). The annotations were manually done by crowdsourcing.\u201d<\/p>\n\n\n\n To understand this, explore the Add the following line to your R script and run it, to see the data frame generated from the previous execution of the The results should look like Figure 10.<\/p>\n\n\n\n <\/p>\n\n\n\n Figure 10. Data frame returned by get_nrc_sentiment function<\/strong><\/p>\n\n\n\n The output shows that the first line of text has;<\/p>\n\n\n The next step is to create two plots charts to help visually analyze the emotions in this survey text. First, perform some data transformation and clean-up steps before plotting charts. The first plot shows the total number of instances of words in the text, associated with each of the eight emotions. Add the following code to your R script and run it.<\/p>\n\n\n\n You can see the bar plot in Figure 11.<\/p>\n\n\n\n <\/p>\n\n\n\n Figure 11. Bar Plot showing the count of words in the text, associated with each emotion<\/strong><\/p>\n\n\n\n This bar chart demonstrates that words associated with the positive emotion of \u201ctrust\u201d occurred about five hundred times in the text, whereas words associated with the negative emotion of \u201cdisgust\u201d occurred less than 25 times. A deeper understanding of the overall emotions occurring in the survey response can be gained by comparing these number as a percentage of the total number of meaningful words. Add the following code to your R script and run it.<\/p>\n\n\n\n The Emotions bar plot can be seen in figure 12.<\/p>\n\n\n\n <\/p>\n\n\n\n Figure 12. Bar Plot showing the count of words associated with each sentiment expressed as a percentage <\/strong><\/p>\n\n\n\n This bar plot allows for a quick and easy comparison of the proportion of words associated with each emotion in the text. The emotion \u201ctrust\u201d has the longest bar and shows that words associated with this positive emotion constitute just over 35% of all the meaningful words in this text. On the other hand, the emotion of \u201cdisgust\u201d has the shortest bar and shows that words associated with this negative emotion constitute less than 2% of all the meaningful words in this text. Overall, words associated with the positive emotions of \u201ctrust\u201d and \u201cjoy\u201d account for almost 60% of the meaningful words in the text, which can be interpreted as a good sign of team health.<\/p>\n\n\n\n This article demonstrated reading text data into R, data cleaning and transformations. It demonstrated how to create a word frequency table and plot a word cloud, to identify prominent themes occurring in the text. Word association analysis using correlation, helped gain context around the prominent themes. It explored four methods to generate sentiment scores, which proved useful in assigning a numeric value to strength (of positivity or negativity) of sentiments in the text and allowed interpreting that the average sentiment through the text is trending positive. Lastly, it demonstrated how to implement an emotion classification with NRC sentiment and created two plots to analyze and interpret emotions found in the text.<\/p>\n\n\n\n Install the syuzhet package (install.packages(‘syuzhet’)). Load a text vector into R, then call get_sentiment(text_vector, method = ‘afinn’) for a numeric sentiment score per sentence (positive = positive, negative = negative), or get_nrc_sentiment(text_vector) for NRC emotion classification (joy, trust, fear, surprise, sadness, disgust, anger, anticipation). For a corpus of documents, use the tm package to clean the text first, then apply syuzhet to the cleaned data. The result is a data frame of sentiment scores that can be visualised with ggplot2.<\/p>\n <\/div>\n Core text mining packages: tm (Corpus creation, text cleaning – removes punctuation, numbers, stop words, whitespace); SnowballC (word stemming to reduce words to their root form); RColorBrewer and wordcloud (word cloud visualisation); ggplot2 (sentiment and frequency bar charts); syuzhet (sentiment scores and NRC emotion classification). Install all with: install.packages(c(‘tm’, ‘SnowballC’, ‘wordcloud’, ‘RColorBrewer’, ‘ggplot2’, ‘syuzhet’)). For reading various file formats: readr for CSV, readtext for Word documents and PDFs.<\/p>\n <\/div>\n The NRC Emotion Lexicon (EmoLex) is a crowd-sourced lexicon of English words and their associations with eight basic emotions (anger, fear, anticipation, trust, surprise, sadness, joy, disgust) and two sentiments (positive, negative). Each word is tagged with 0 or 1 for each emotion\/sentiment category. In R, the syuzhet package’s get_nrc_sentiment() function looks up each word in the text against the NRC lexicon and returns counts for each emotion category. It provides richer analysis than positive\/negative scoring alone – useful for understanding the emotional tenor of customer feedback or survey responses.<\/p>\n <\/div>\n Load the wordcloud and tm packages. Create a Corpus from your text data, clean it (tm_map for punctuation, stop words, whitespace), build a TermDocumentMatrix, and convert to a matrix. Sort words by frequency: word_freq = sort(rowSums(as.matrix(tdm)), decreasing = TRUE). Call wordcloud(words = names(word_freq), freq = word_freq, min.freq = 3, random.order = FALSE, colors = brewer.pal(8, ‘Dark2’)). Adjust min.freq to control how many words appear and scale for size parameters based on your dataset size.<\/p>\n <\/div>\n <\/section>\n","protected":false},"excerpt":{"rendered":" Text mining and sentiment analysis with R: install the tm package, clean text data, build term-document matrices, generate word clouds, analyse word associations, calculate sentiment scores, and classify emotions using the NRC lexicon.…<\/p>\n","protected":false},"author":317671,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":true,"footnotes":""},"categories":[143528,47,53,1],"tags":[95509],"coauthors":[101710],"class_list":["post-87138","post","type-post","status-publish","format-standard","hentry","category-bi-sql-server","category-data-science","category-featured","category-uncategorized","tag-standardize"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/87138","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/users\/317671"}],"replies":[{"embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/comments?post=87138"}],"version-history":[{"count":12,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/87138\/revisions"}],"predecessor-version":[{"id":109850,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/87138\/revisions\/109850"}],"wp:attachment":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/media?parent=87138"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/categories?post=87138"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/tags?post=87138"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/coauthors?post=87138"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}\n
![\"A](\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2020\/05\/a-screenshot-of-a-computer-description-automatica.png\")
<\/figure>\n\n\n\nInstalling and loading R packages<\/h2>\n\n\n\n
\n
# Install\ninstall.packages(\"tm\") # for text mining\ninstall.packages(\"SnowballC\") # for text stemming\ninstall.packages(\"wordcloud\") # word-cloud generator \ninstall.packages(\"RColorBrewer\") # color palettes\ninstall.packages(\"syuzhet\") # for sentiment analysis\ninstall.packages(\"ggplot2\") # for plotting graphs\n# Load\nlibrary(\"tm\")\nlibrary(\"SnowballC\")\nlibrary(\"wordcloud\")\nlibrary(\"RColorBrewer\")\nlibrary(\"syuzhet\")\nlibrary(\"ggplot2\")<\/pre>\n\n\n\n
Reading file data into R<\/h2>\n\n\n\n
read.table()<\/code> is generally used to read a file in table format and imports data as a data frame. Several variants of this function are available, for importing different file formats;<\/p>\n\n\n\n
readLines<\/code> function. This function takes a file (or URL) as input and returns a vector containing as many elements as the number of lines in the file. The readLines<\/code> function simply extracts the text from its input source and returns each line as a character string. The n=<\/code> argument is useful to read a limited number (subset) of lines from the input source (Its default value is -1, which reads all lines from the input source). When using the filename in this function\u2019s argument, R assumes the file is in your current working directory (you can use the getwd()<\/code> function in R console to find your current working directory). You can also choose the input file interactively, using the file.choose()<\/code> function within the argument. The next step is to load that Vector as a Corpus. In R, a Corpus is a collection of text document(s) to apply text mining or NLP routines on. Details of using the readLines<\/code> function are sourced from: https:\/\/www.stat.berkeley.edu\/~spector\/s133\/Read.html<\/a> .<\/p>\n\n\n\n# Read the text file from local machine , choose file interactively\ntext <- readLines(file.choose())\n# Load the data as a corpus\nTextDoc <- Corpus(VectorSource(text))<\/pre>\n\n\n\n
![\"A](\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2020\/05\/a-screenshot-of-a-computer-description-automatica-1.png\")
<\/figure>\n\n\n\nCleaning up Text Data<\/h2>\n\n\n\n
tm_map()<\/code> function to replace special characters like \/<\/code>, @<\/code> and |<\/code> with a space. The next step is to remove the unnecessary whitespace and convert the text to lower case.<\/p>\n\n\n\nstopwords<\/code> in the tm_map()<\/code> function supports several languages like English, French, German, Italian, and Spanish. Please note the language names are case sensitive. I will also demonstrate how to add your own list of stopwords, which is useful in this Team Health example for removing non-default stop words like \u201cteam\u201d, \u201ccompany\u201d, \u201chealth\u201d. Next, remove numbers and punctuation.<\/p>\n\n\n\n #Replacing \"\/\", \"@\" and \"|\" with space\ntoSpace <- content_transformer(function (x , pattern ) gsub(pattern, \" \", x))\nTextDoc <- tm_map(TextDoc, toSpace, \"\/\")\nTextDoc <- tm_map(TextDoc, toSpace, \"@\")\nTextDoc <- tm_map(TextDoc, toSpace, \"\\\\|\")\n# Convert the text to lower case\nTextDoc <- tm_map(TextDoc, content_transformer(tolower))\n# Remove numbers\nTextDoc <- tm_map(TextDoc, removeNumbers)\n# Remove english common stopwords\nTextDoc <- tm_map(TextDoc, removeWords, stopwords(\"english\"))\n# Remove your own stop word\n# specify your custom stopwords as a character vector\nTextDoc <- tm_map(TextDoc, removeWords, c(\"s\", \"company\", \"team\")) \n# Remove punctuations\nTextDoc <- tm_map(TextDoc, removePunctuation)\n# Eliminate extra white spaces\nTextDoc <- tm_map(TextDoc, stripWhitespace)\n# Text stemming - which reduces words to their root form\nTextDoc <- tm_map(TextDoc, stemDocument)<\/pre>\n\n\n\n
Building the term document matrix<\/h2>\n\n\n\n
TermDocumentMatrix()<\/code> from the text mining package, you can build a Document Matrix \u2013 a table containing the frequency of words.<\/p>\n\n\n\n# Build a term-document matrix\nTextDoc_dtm <- TermDocumentMatrix(TextDoc)\ndtm_m <- as.matrix(TextDoc_dtm)\n# Sort by descearing value of frequency\ndtm_v <- sort(rowSums(dtm_m),decreasing=TRUE)\ndtm_d <- data.frame(word = names(dtm_v),freq=dtm_v)\n# Display the top 5 most frequent words\nhead(dtm_d, 5)<\/pre>\n\n\n\n
head<\/code> command on RStudio Console.<\/p>\n\n\n\n![\"\"](\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2020\/05\/word-image-1.png\")
<\/figure>\n\n\n\n# Plot the most frequent words\nbarplot(dtm_d[1:5,]$freq, las = 2, names.arg = dtm_d[1:5,]$word,\n col =\"lightgreen\", main =\"Top 5 most frequent words\",\n ylab = \"Word frequencies\")<\/pre>\n\n\n\n
![\"A](\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2020\/05\/a-screenshot-of-a-cell-phone-description-automati.png\")
<\/figure>\n\n\n\n\n
Generate the Word Cloud<\/h2>\n\n\n\n
#generate word cloud\nset.seed(1234)\nwordcloud(words = dtm_d$word, freq = dtm_d$freq, min.freq = 5,\n max.words=100, random.order=FALSE, rot.per=0.40, \n colors=brewer.pal(8, \"Dark2\"))<\/pre>\n\n\n\n
\n
![\"A](\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2020\/05\/a-screenshot-of-a-cell-phone-description-automati-1.png\")
<\/figure>\n\n\n\nWord Association<\/h2>\n\n\n\n
# Find associations \nfindAssocs(TextDoc_dtm, terms = c(\"good\",\"work\",\"health\"), corlimit = 0.25)<\/pre>\n\n\n\n
![\"A](\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2020\/05\/a-screenshot-of-a-cell-phone-description-automati-2.png\")
<\/figure>\n\n\n\ncorlimit = 0.25<\/code> is the lower limit\/threshold I have set. You can set it lower to see more words, or higher to see less). The output indicates that \u201cintegr\u201d (which is the root for word \u201cintegrity\u201d) and \u201csynergi\u201d (which is the root for words \u201csynergy\u201d, \u201csynergies\u201d, etc.) and occur 28% of the time with the word \u201cgood\u201d. You can interpret this as the context around the most frequently occurring word (\u201cgood\u201d) is positive. Similarly, the root of the word \u201ctogether\u201d is highly correlated with the word \u201cwork\u201d. This indicates that most responses are saying that teams \u201cwork together\u201d and can be interpreted in a positive context.<\/p>\n\n\n\n# Find associations for words that occur at least 50 times\nfindAssocs(TextDoc_dtm, terms = findFreqTerms(TextDoc_dtm, lowfreq = 50), corlimit = 0.25)<\/pre>\n\n\n\n
![\"A](\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2020\/05\/a-screenshot-of-a-cell-phone-description-automati-3.png\")
<\/figure>\n\n\n\nSentiment Scores<\/h2>\n\n\n\n
get_sentiment<\/code> function accepts two arguments: a character vector (of sentences or words) and a method. The selected method determines which of the four available sentiment extraction methods will be used. The four methods are syuzhet<\/code> (this is the default), bing<\/code>, afinn<\/code> and nrc<\/code>. Each method uses a different scale and hence returns slightly different results. Please note the outcome of nrc<\/code> method is more than just a numeric score, requires additional interpretations and is out of scope for this article. The descriptions of the get_sentiment<\/code> function has been sourced from : https:\/\/cran.r-project.org\/web\/packages\/syuzhet\/vignettes\/syuzhet-vignette.html?<\/a><\/p>\n\n\n\n# regular sentiment score using get_sentiment() function and method of your choice\n# please note that different methods may have different scales\nsyuzhet_vector <- get_sentiment(text, method=\"syuzhet\")\n# see the first row of the vector\nhead(syuzhet_vector)\n# see summary statistics of the vector\nsummary(syuzhet_vector)<\/pre>\n\n\n\n
![\"A](\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2020\/05\/a-screenshot-of-a-cell-phone-description-automati-4.png\")
<\/figure>\n\n\n\nsyuzhet<\/code> method is decimal and ranges from -1(indicating most negative) to +1(indicating most positive). Note that the summary statistics of the suyzhet<\/code> vector show a median value of 1.6, which is above zero and can be interpreted as the overall average sentiment across all the responses is positive.<\/p>\n\n\n\n# bing\nbing_vector <- get_sentiment(text, method=\"bing\")\nhead(bing_vector)\nsummary(bing_vector)\n#affin\nafinn_vector <- get_sentiment(text, method=\"afinn\")\nhead(afinn_vector)\nsummary(afinn_vector)<\/pre>\n\n\n\n
![\"A](\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2020\/05\/a-screenshot-of-a-cell-phone-description-automati-5.png\")
<\/figure>\n\n\n\n\n
bing<\/code> and afinn<\/code> vectors also show that the Median<\/code> value of Sentiment scores is above 0 and can be interpreted as the overall average sentiment across the all the responses is positive.<\/p>\n\n\n\nsign<\/code> function, which converts all positive number to 1, all negative numbers to -1 and all zeros remain 0.<\/p>\n\n\n\n#compare the first row of each vector using sign function\nrbind(\n sign(head(syuzhet_vector)),\n sign(head(bing_vector)),\n sign(head(afinn_vector))\n)<\/pre>\n\n\n\n
![\"A](\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2020\/05\/a-screenshot-of-a-cell-phone-description-automati-6.png\")
<\/figure>\n\n\n\nEmotion Classification<\/h2>\n\n\n\n
get_nrc_sentiments<\/code> function, which returns a data frame with each row representing a sentence from the original file. The data frame has ten columns (one column for each of the eight emotions, one column for positive sentiment valence and one for negative sentiment valence). The data in the columns (anger, anticipation, disgust, fear, joy, sadness, surprise, trust, negative, positive) can be accessed individually or in sets. The definition of get_nrc_sentiments<\/code> has been sourced from: https:\/\/cran.r-project.org\/web\/packages\/syuzhet\/vignettes\/syuzhet-vignette.html?<\/a><\/p>\n\n\n\nget_nrc_sentiment<\/code> function.<\/p>\n\n\n\n# run nrc sentiment analysis to return data frame with each row classified as one of the following\n# emotions, rather than a score: \n# anger, anticipation, disgust, fear, joy, sadness, surprise, trust \n# It also counts the number of positive and negative emotions found in each row\nd<-get_nrc_sentiment(text)\n# head(d,10) - to see top 10 lines of the get_nrc_sentiment dataframe\nhead (d,10)<\/pre>\n\n\n\n
![\"A](\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2020\/05\/a-screenshot-of-a-cell-phone-description-automati-7.png\")
<\/figure>\n\n\n\n\n
#transpose\ntd<-data.frame(t(d))\n#The function rowSums computes column sums across rows for each level of a grouping variable.\ntd_new <- data.frame(rowSums(td[2:253]))\n#Transformation and cleaning\nnames(td_new)[1] <- \"count\"\ntd_new <- cbind(\"sentiment\" = rownames(td_new), td_new)\nrownames(td_new) <- NULL\ntd_new2<-td_new[1:8,]\n#Plot One - count of words associated with each sentiment\nquickplot(sentiment, data=td_new2, weight=count, geom=\"bar\", fill=sentiment, ylab=\"count\")+ggtitle(\"Survey sentiments\")<\/pre>\n\n\n\n
![\"A](\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2020\/05\/a-screenshot-of-a-cell-phone-description-automati-8.png\")
<\/figure>\n\n\n\n#Plot two - count of words associated with each sentiment, expressed as a percentage\nbarplot(\n sort(colSums(prop.table(d[, 1:8]))), \n horiz = TRUE, \n cex.names = 0.7, \n las = 1, \n main = \"Emotions in Text\", xlab=\"Percentage\"\n)<\/pre>\n\n\n\n
![\"A](\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2020\/05\/a-screenshot-of-a-cell-phone-description-automati-9.png\")
<\/figure>\n\n\n\nConclusion<\/h2>\n\n\n\n
References:<\/h2>\n\n\n
\n
FAQs: Text Mining and Sentiment Analysis: Analysis with R<\/h2>\n\n
1. How do I do sentiment analysis in R?<\/h3>\n
2. What R packages do I need for text mining?<\/h3>\n
3. What is the NRC Word-Emotion Association Lexicon?<\/h3>\n
4. How do I generate a word cloud in R from text data?<\/h3>\n