Now, we continue what we started in (Sentiment Analysis-Guide to Call Center Performance - Part 1), and implement a really cool step, which is sentiment analysis.
Sentiment analysis
Sentiment analysis is among the most exciting problems in the field of NLP (Natural Language Processing). It consists of the identification of a text attitude towards a topic, positively or negatively, and returning a score between some maximum negative and maximum positive (e.g., 0 to 1 or -1 to 1). The sentiment analysis API depends on detecting proper and common nouns. Typically, the lowest value refers to a very negative attitude and the highest refers to a very positive attitude. Usually, a particular threshold is set to identify positive and negative sentiment (e.g., 70% and 30% from the range respectively).
Sentiment analysis produces a higher quality result when you give it smaller chunks of text to work on. This is opposite from key phrase extraction, which performs better on larger blocks of text. To get the best results from both operations, consider restructuring the inputs accordingly.
As expected, all vendors have their version of sentiment analysis. Microsoft has sentiment analysis with Azure Text Analytics suite, Google in Cloud Natural Language API, Amazon in Amazon Comprehend and IBM has Watson tone analyzer.
On the surface, all APIs might seem similar, and it is just a matter of pick and choose. As a matter, if fact, this is incorrect, there are crucial differences that can significantly impact your analysis results. Let us have a quick discussion.
Azure Text Analytics: I would consider Microsoft offering is the simplest in terms of capabilities. The API only returns” score” parameter which ranges from zero (very negative) to one (very positive). Therefore, it does not provide any context. A common problem with such simplistic approach is that it confuses a neutral statement (score around 0.5) with a statement that contains both very positive and negative sentiments which will cancel out and generate a sentiment with a score around 0.5 too. A sample response is shown below.
{
"documents": [
{
"score": 0.9999237060546875,
"id": "1"
},
{
"score": 0.0000540316104888916,
"id": "2"
},
{
"score": 0.99990355968475342,
"id": "3"
},
{
"score": 0.980544924736023,
"id": "4"
},
{
"score": 0.99996328353881836,
"id": "5"
}
],
"errors": []
}
Google in Cloud Natural Language API: Google fixes the confusion problem Microsoft API has by introducing the magnitude parameter. The magnitude parameter is an unnormalized parameter that lies between zero and infinity and indicates the strength of the expression (positive or negative), typically longer text blocks will result in greater magnitude. A neutral sentiment will result in an average score, and very low magnitude, and a mixed sentiment will result in an average score and a high magnitude. A sample response is shown below.
{
"documentSentiment": {
"magnitude": 0.8,
"score": 0.8
},
"language": "en",
"sentences": [
{
"text": {
"content": "Enjoy your vacation!",
"beginOffset": 0
},
"sentiment": {
"magnitude": 0.8,
"score": 0.8
}
}
]
}Amazon Comprehend sentiment analysis service: Amazon follows a more explicit approach to differentiate sentiments, by breaking down each sentiment type with its corresponding confidence level. A sample response is shown below.
{
“SentimentScore”: {
“Mixed”: 0.030585512690246105,
“Positive”: 0.94992071056365967,
“Neutral”: 0.0141543131828308,
“Negative”: 0.00893945890665054
},
“Sentiment”: “POSITIVE”,
“LanguageCode”: “en”
}IBM Watson Tone Analyzer: I would consider this is the most interesting service. The service comes in two flavors: i) General purpose and ii) Customer engagement. Watson tone analyzer does not return negative or positive sentiments, but rather a description of the tone. I would consider Watson tone analyzer is the most suitable option to understand customer emotions.
General-purpose endpoint informs whether the tone is anger, fear, joy, sadness, analytical, confident, or tentative. While the customer engagement endpoint informs if the tone is excited, frustrated, impolite, polite, sad, satisfied or sympathetic. Sample responses are shown below.
{
"document_tone": {
"tones": [
{
"score": 0.6165,
"tone_id": "sadness",
"tone_name": "Sadness"
},
{
"score": 0.829888,
"tone_id": "analytical",
"tone_name": "Analytical"
}
]
},
"sentences_tone": [
{
"sentence_id": 0,
"text": "Team, I know that times are tough!",
"tones": [
{
"score": 0.801827,
"tone_id": "analytical",
"tone_name": "Analytical"
}
]
},
{
"sentence_id": 1,
"text": "Product sales have been disappointing for the past three quarters.",
"tones": [
{
"score": 0.771241,
"tone_id": "sadness",
"tone_name": "Sadness"
},
{
"score": 0.687768,
"tone_id": "analytical",
"tone_name": "Analytical"
}
]
},
{
"sentence_id": 2,
"text": "We have a competitive product, but we need to do a better job of selling it!",
"tones": [
{
"score": 0.506763,
"tone_id": "analytical",
"tone_name": "Analytical"
}
]
}
]
}
{
"utterances_tone": [
{
"utterance_id": 0,
"utterance_text": "Hello, I'm having a problem with your product.",
"tones": [
{
"score": 0.686361,
"tone_id": "polite",
"tone_name": "Polite"
}
]
},
{
"utterance_id": 1,
"utterance_text": "OK, let me know what's going on, please.",
"tones": [
{
"score": 0.92724,
"tone_id": "polite",
"tone_name": "Polite"
}
]
}
]
}
Some factors to consider when choosing sentiment analysis API
Before opting to a particular API, I would recommend asking the following questions:
- What content limit is imposed by the API (e.g. number of characters/bytes)
- What languages are supported?
- Does it support batch operations?
- Does it support custom learning (e.g. providing some domain-specific terms)?
- Does it support language detection?
- Does it support asynchronous operations?
Finally, my original plan was to create a demo using C# code, however, I provide a short youtube video soon showing how to use the APIs with API console so that we learn something new : - )
