Using Cloud-Based AI Keyword Analysis for Text Optimization
Recently I’ve seen a lot of discussion and debate about the use of Applicant Tracking Systems and how they parse resumes for keywords to try to determine if a candidate is a good fit for a job posting. Regardless of one’s thoughts on the systems, for better or for worse, the reality is they are widely used today. Using public cloud-based artificial intelligence I was curious to see if you could use these online services to try to better your odds at matching for keyphrases an employer or recruiter might use for jobs you’d be interested in applying for. To put each companies respective engines to the test and have a better idea for the strengths and weaknesses of this technology I also ran a simple tweet, a newspaper editorial, and the bulk of a poem through each. I was somewhat surprised at the outcomes.
I’ll go into specifics in the results section of each service but all in all I most pleased with the results from IBM Watson. Surprisingly Azure came in second best for me with Google and AWS falling short in my opinion.
You may also notice some similarity with a previous post, Detecting Sentiment and Emotion in Text, but that’s unavoidable as the keyword or keyphrase functionality is tied in with all the related natural language processing services. However, these functions serve very different purposes. What the text is about rather than how the text may make one feel. Although, some of the services will return sentiment along with their keywords as I’ll note where appropriate.
Below are the 4 samples of text I used to test each of the platforms’ artificial intelligence offerings:
Sample #1 is from my LinkedIn bio:
I am a well-rounded, cloud focused, IT professional. I’m comfortable in any role from contributor to manager. I’m willing to grow within an organization to achieve the goal of returning to a manager/director role in a cloud-focused company.
My roles and interests encompass a wide range of responsibilities that provide me with expertise in many areas including cloud solutions offered by Amazon Web Services and Microsoft Azure, security, domains and DNS, web technologies (servers and development), and legal compliance. My experience also includes responsibilities for developing code to interact with cloud platform APIs, Iaas, DaaS, HPE Helion, CloudSystem 9 and 10, Scality, Stratoscale Symphony, Google Analytics, Google Adwords, etc.
Throughout my 10 years of leadership positions some of the numerous initiatives I have been responsible for include:
• Collecting and correlating data to design an anti-fraud solution that decreased fraud by 86%
• Reorganized and led a team of 45 agents increasing service levels from 45% to 90% in less than 6 weeks
• The design & implementation of a global load balancing solution that improved page load times by up to 500% depending on global locationProgram Management and technical project management has allowed me to work with world-class customers delivering complex multi-million-dollar solutions on a timely basis. In doing so, I work with numerous vendors and inside team members from executive management to professional services, engineers, and developers.
Sample #2 is the first nearly 5000 characters of Edgar Allen Poe’s “The Raven“.
Sample #3 is the text of an editorial on cryptocurrencies by The Guardian.
Sample #4 is this tweet about a cute dog:
Meet Yogi. He made his first trip to the beach today. Could never have guessed eating sand would be so exhausting. 12/10 would bring back soon pic.twitter.com/8dnaCObcZ4
— WeRateDogs™ (@dog_rates) February 19, 2018
Let’s pull back the curtain and see what each service reports for our examples.
AWS Comprehend
Features
AWS offers the following features as part of its Comprehend natural language processing service. In addition to the overall sentiment detected, the Sentiment Analysis function will give you scores for each possible value to show you how certain it is of its decision, out to 14 decimal places. I’ll be rounding that up a bit for the sake of your eyes. For this post, I only experimented with the Keyphrase Extraction function.
Keyphrase Extraction: The Keyphrase Extraction API returns the key phrases or talking points and a confidence score to support that this is a key phrase.
Sentiment Analysis: The Sentiment Analysis API returns the overall sentiment of a text (Positive, Negative, Neutral, or Mixed).
Entity Recognition: The Entity Recognition API returns the named entities (“People,” “Places,” “Locations,” etc.) that are automatically categorized based on the provided text.
Language Detection: The Language Detection API automatically identifies text written in over 100 languages and returns the dominant language with a confidence score to support that a language is dominant.
Topic Modeling: Topic Modeling identifies relevant terms or topics from a collection of documents stored in Amazon S3. It will identify the most common topics in the collection and organize them in groups and then map which documents belong to which topic.
Pricing
Natural Language Processing requests are measured in units of 100 characters, with a 3 unit (300 characters) minimum charge per request. The free tier gives you access to 50,000 units of text (about 5 million characters) for each of the APIs per month with each additional unit starting at $0.0001 with significant cost reductions based on volume. Topic Modeling is the exception as it is priced per job.
I like Amazon’s price structure as it gives you quite a lot of wiggle room to play and remains inexpensive to continue using. Complete pricing can be found via the appropriate link in the Resources box above.
Limits
For my tests, only the first limit really applied. It was sufficient for my needs and seems large enough to accommodate most tasks you’d need it for. I included all limits, as of writing this post, for reference:
- The maximum document size is 5,000 bytes of UTF-8 encoded characters.
- The maximum number of documents for the BatchDetectDominantLanguage, BatchDetectEntities, BatchDetectKeyPhrases, and BatchDetectSentiment operations is 25 documents per request.
- The BatchDetectDominantLanguage and DetectDominantLanguage operations have the following limitations:
- They don’t support phonetic language detection. For example, they will not detect “arigato” as Japanese, nor “nihao” as Chinese.
- They may have trouble distinguishing close language pairs, such as Indonesian and Malay; or Bosnian, Croatian, and Serbian.
- For best results, the input text should be at least 20 characters long.
API
With Amazon, you really have to install and use either the CLI or language-specific SDK. I used the PHP SDK and found it really easy to install and quick to code against – not to mention clean. For the authentication in the SDK to be successful please be sure your system’s clock/time is set correctly.
<?php
// Setup and install your SDK first! For PHP you do that with composer
require 'vendor/autoload.php';
// Provide you AWS API keys, the user for these keys need to be given permissions to Comprehend
$aws_access_key = '';
$aws_secret_access_key = '';
// The text you want to analyze
$text = '';
// Do the work and get your result
$aws = new Aws\Sdk([
'version' => 'latest',
'region' => 'us-east-1',
'credentials' => [
'key' => $aws_access_key,
'secret' => $aws_secret_access_key,
],
'Comprehend' => [
'region' => 'us-east-1'
],
]);;
$comprehend = $aws->createComprehend();
$aws_result = $comprehend->detectKeyPhrases([
'LanguageCode' => 'en',
'Text' => $text
]);
// $aws_result will be an array you can access for the returned data
echo 'AWS Results' . PHP_EOL;
echo 'Keyphrases: ' . PHP_EOL;
foreach ( $aws_result['KeyPhrases'] as $i ) {
echo $i['Text'] . ' (' . $i['Score'] . ')' . PHP_EOL;
}
// Alternately, if you'd like to see the full set of data returned
print_r($aws_result);
?>
Results
I found AWS seems to just run through each sentence and pick out each keyword it runs across. This results in a lot of keyword duplication. That’s fine if you are looking at relevance within a sentence or a small block of text but I personally would be looking at the document as a whole rather than chunked. I also note that Comprehend tends to be very loose on keyphrase detection (“Amazon Web Services and Micrsoft Azure”), tends to include some punctuation in keywords (see sample #2), and some words or phrases that I would have picked out either were not returned as important or had low confidence scores.
As mentioned above I’ve rounded the 14 decimal places up for readability. You get back a collection of key phrases that Amazon Comprehend identified from the input text. For each key phrase, the response provides the text of the keyword or phrase, where the keyphrase begins and ends, and the level of confidence that Amazon Comprehend has in the accuracy of the detection. I believe the score is from 0 to 1.0 with higher values indicating a higher confidence but I didn’t see this specifically mentioned in the documentation; I’m basing it on how AWS documents sentiment scores on other functions in the Comprehend service. Below I only list the first 20 keyphrases for each sample along with the confidence score for each.
| Sample #1 – LinkedIn Bio | Sample #2 – The Raven | Sample #3 – Editorial | Sample #4 – Dog Tweet |
| a well-rounded, cloud focused (0.895357) | a midnight dreary (0.856855) | Last month (0.990694) | Yogi (0.779558) |
| any role (0.991994) | many a quaint and curious volume (0.884559) | a plague (0.998621) | his first trip (0.999373) |
| contributor (0.986457) | forgotten lore (0.920504) | kittens (0.999314) | the beach (0.932810) |
| manager (0.988315) | nearly napping (0.721991) | the most fashionable cryptocurrencies (0.998065) | today (0.994661) |
| an organization (0.9967615) | a tapping (0.996199) | the internet (0.995693) | sand (0.915146) |
| the goal (0.999682) | some one (0.849618) | news (0.983833) | 12/10 (0.981981) |
| a manager/director role (0.9908485) | my chamber door (0.881901) | the cryptocurrency (0.72025179862976) | |
| a cloud-focused company (0.997656) | “’Tis (0.746844) | Ethereum (0.933024) | |
| My roles and interests (0.888028) | some visitor (0.985374) | bills (0.744276) | |
| a wide range (0.999404) | my chamber door (0.898626) | the world computer” (0.847476) | |
| responsibilities (0.999585) | Only this (0.714963) | a distributed program (0.993759) | |
| expertise (0.996745) | the bleak December (0.984577) | large parts (0.999525) | |
| many areas (0.998761) | each separate dying ember (0.912081) | both the legitimate banking system (0.948817) | |
| cloud solutions (0.998675) | its ghost (0.998404) | the legal system (0.998754) | |
| Amazon Web Services and Microsoft Azure (0.934452) | the floor (0.996622) | contracts (0.934160) | |
| security (0.934451) | the morrow (0.996977) | computer code (0.997170) | |
| domains (0.898301) | my books surcease (0.957688) | Ethereum (0.986565) | |
| DNS (0.667329) | sorrowsorrow (0.996818) | the plaything (0.997950) | |
| web technologies (0.992197) | skip all the way down to spot 69… | skip down to spot 115, the last items returned… | |
| servers and development (0.856500) | a stately Raven (0.879681) | the cryptokittens (0.977406) |
Azure Text Analytics
| Resources: |
| Text Analytics Product Page |
| Text Analytics Documentation |
| Text Analytics API Docs |
| Text Analytics Pricing |
Features
Azure’s feature set in Text Analytics is considerably lighter than the competing services. It does what I was looking for so I can’t complain about that but if you’re looking for more functionality you may need to look at IBM or AWS.
Sentiment analysis
The API returns a numeric score between 0 and 1. Scores close to 1 indicate positive sentiment, and scores close to 0 indicate negative sentiment. Sentiment score is generated using classification techniques. The input features of the classifier include n-grams, features generated from part-of-speech tags, and word embeddings. It is supported in a variety of languages.
Key phrase extraction
The API returns a list of strings denoting the key talking points in the input text. We employ techniques from Microsoft Office’s sophisticated Natural Language Processing toolkit. English, German, Spanish, and Japanese text are supported.
Language detection
The API returns the detected language and a numeric score between 0 and 1. Scores close to 1 indicate 100% certainty that the identified language is true. A total of 120 languages are supported.
Pricing
The free tier gives you up to 5,000 transactions. Once you go beyond that mark be prepared to pay $75 for the next tier. There is no per transaction incremental charge so you’ll see big jumps in fees for potentially small changes in your usage. Each “document” analyzed for each API call will be considered a transaction. Honestly, the subscription model they offer would likely keep me from using Azure’s Text Analytics for any small projects that I didn’t have an immediate method to monetize.
Limits
I do like that Azure will let you push a lot of text through at one time to help reduce the likelihood of being rate limited. Their “document” size is in-line with Amazon’s limits as well.
- Maximum size of a single document 5,000 characters as measured by String.Length.
- Maximum size of entire request 1 MB
- Maximum number of documents in a request 1,000 documents
- The rate limit is 100 calls per minute. Note that you can submit a large quantity of documents in a single call (up to 1000 documents).
API
With a little Googling, I found a PHP SDK for Azure but it seems to only support the base IaaS services right now. I was going for consistency with using a single programming language in this post so I had to use Azure’s REST API. Authentication is handled via passing your API key via a header in your HTTP POST.
<?php
// Provide your Azure API key to access the Text Analytics service
$azure_key_1 = '';
$azure_endpoint = 'https://eastus.api.cognitive.microsoft.com/text/analytics/v2.0/keyPhrases';
// The text you want to analyze
$text = '';
// Do the work and get your result
$data = array(
'documents' => array(
array(
'language' => 'en',
'id' => '1',
'text' => "$text"
)
)
);
$data = json_encode($data);
$azure = curl_init();
curl_setopt($azure, CURLOPT_URL, $azure_endpoint);
curl_setopt($azure, CURLOPT_POST, 1);
curl_setopt($azure, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($azure, CURLOPT_HTTPHEADER, array(
"Ocp-Apim-Subscription-Key: $azure_key_1",
'Content-Type: application/json',
'Accept: application/json'
));
curl_setopt($azure, CURLOPT_POSTFIELDS, $data);
$response = curl_exec($azure);
$azure_result = json_decode($response, true);
curl_close($azure);
// $azure_result will be an array you can access for the returned data
echo 'Azure Results:' . PHP_EOL;
foreach ( $azure_result['documents'][0]['keyPhrases'] as $i ) {
echo $i . PHP_EOL;
}
// Alternately, if you'd like to see the full set of data returned
print_r($azure_result);
?>
Results
For the sentiment tests, I wasn’t crazy about lack of detail from Azure. However, in this instance, I’m relatively happy with their keyword results. I received the list of keyphrases I was looking for and, in my opinion, their parsing is fairly tight and pretty accurate as far as what I would pick out. However, I felt some interesting keywords were missed or too broadly lumped in as a phrase (I’m thinking about ravens and cryptokittens here). They stripped out irrelevant punctuation from words and seemingly avoided duplication, which I liked. Don’t get me wrong, some results are still a little odd but at least Azure doesn’t break Amazon away from Web Services or lump it in as part of a larger phrase. I’m only listing the first 20 keywords returned for each sample.
| Sample #1 – LinkedIn Bio | Sample #2 – The Raven | Sample #3 – Editorial | Sample #4 – Dog Tweet |
| cloud solutions | chamber door | code of Ethereum | trip |
| cloud-focused company | tapping | software | beach |
| professional services | ominous bird of yore | week | sand |
| executive management | ebony bird | ignorance of computer code | Yogi |
| cloud platform APIs | lost Lenore | world computer | |
| Program Management | memories of Lenore | wake of bitcoin | |
| fraud solution | stately Raven | fashionable cryptocurrencies | |
| manager | whispered word | thousands of cryptocurrencies | |
| design | sad fancy | legitimate banking system | |
| global load balancing solution | Tis | legal system | |
| Amazon Web Services | soul | modern computer chips | |
| technical project management | ancient Raven wandering | real currency | |
| web technologies | sculptured bust | trust | |
| Google Analytics | placid bust | times | |
| dollar solutions | bust of Pallas | propagation of imaginary kittens | |
| team members | angels | legitimate companies | |
| Google Adwords | entrance | world’s credit card system | |
| director role | velvet-violet lining | medium of exchange | |
| numerous initiatives | melancholy burden bore | excellent medium | |
| numerous vendors | cushion’s velvet lining | plague of kittens |
Google Natural Language
| Resources: |
| Natural Language Product Page |
| Natural Language Documentation |
| Natural Language REST API Docs |
| Natural Language Pricing |
Features
Google offers some interesting tools. Syntax Analysis and Entity Recognition look pretty cool although I’m not sure what practical use I would personally have for them. I had to read through the docs a bit to catch they lump keyword detection in with Entity Recognition.
Syntax Analysis
Extract tokens and sentences, identify parts of speech (PoS) and create dependency parse trees for each sentence.
Entity Recognition
Identify entities and label by types such as person, organization, location, events, products, and media.
Sentiment Analysis
Understand the overall sentiment expressed in a block of text.
Content Classification
Classify documents in predefined 700+ categories.
Multi-Language
Enables you to easily analyze text in multiple languages including English, Spanish, Japanese, Chinese (Simplified and Traditional), French, German, Italian, Korean and Portuguese.
Pricing
The Natural Language API is priced using units of measurement known as text records. A text record may contain up to 1,000 Unicode characters within the text content sent to the API for evaluation. Text in excess of these 1,000 characters counts as additional records. Prices are expressed in dollars per 1,000 text records (1,000,000 Unicode characters).
- Free up to 5K
- Over 5K “text records” processed cost depends on features used.
Limits
Google’s limits are based on text size, words/tokens in the text, and entity mentions. The API responds in different ways based on which limit you’ve exceeded. It’s not easy to describe so I’d recommend you read up on Googles Natural Language Quotas.
API
Google Cloud Platform does offer a PHP SDK but frankly, there were too many packages required to be installed to support it on my VM and I didn’t want to hassle with it. I decided to simply use their REST API instead. Your requests are authenticated by including your API key in the POST URI.
<?php
// Provide your gcp API key to access the Text Analytics service
$gcp_api_key = '';
$gcp_endpoint = 'https://language.googleapis.com/v1/documents:analyzeEntitySentiment?key=';
// The text you want to analyze
$text = '';
// Do the work and get your result
$data = array(
'document' => array(
'type' => 'PLAIN_TEXT',
'language' => 'en',
'content' => "$text"
),
'encodingType' => 'UTF8'
);
$data = json_encode($data);
$gcp = curl_init();
curl_setopt($gcp, CURLOPT_URL, "$gcp_endpoint$gcp_api_key");
curl_setopt($gcp, CURLOPT_POST, 1);
curl_setopt($gcp, CURLOPT_RETURNTRANSFER, true);
curl_setopt($gcp, CURLOPT_HTTPHEADER, array(
'Content-Type: application/json',
'Accept: application/json'
));
curl_setopt($gcp, CURLOPT_POSTFIELDS, $data);
$response = curl_exec($gcp);
$gcp_result = json_decode($response, true);
curl_close($gcp);
// $gcp_result will be an array you can access for the returned data
echo 'Google Cloud Platform' . PHP_EOL;
echo 'Keyphrases: ' . PHP_EOL;
foreach ( $gcp_result['entities'] as $i ) {
echo $i['name'] . PHP_EOL;
echo 'Salience: ' . $i['salience'] . PHP_EOL;
echo 'Sentiment: ' . PHP_EOL;
echo ' - Score: ' . $i['sentiment']['score'] . PHP_EOL;
echo ' - Magnitude: ' . $i['sentiment']['magnitude'] . PHP_EOL;
echo PHP_EOL;
}
// Alternately, if you'd like to see the full set of data returned
print_r($gcp_result);
?>
Results
Google’s keyword or phrase matching seems very tight, almost too tight. Note “Amazon” is stipped off from “Web Services”. Google breaks your document down to sentence entities and finds keywords in each entity so you will see duplication of keywords and differing salience values because of that. The salience scores seemed a bit off to what I would have thought. Specifically looking at Sample #3, Ethereum had a score of 0.0057588066 and cryptokittens the value of 0.001952143. Both of which I would have expected to be higher.
The Salience value looks to be a value between 0 and 1.0. The higher the relevance of the keyword to the text the higher the salience score will be.
Google’s Score is a value between -1.0 (Negative) and 1.0 (positive) and corresponds to the overall emotional leaning of the text. Strong positive and negative statements could balance each other out in the score. Generally, negative is -1.0 to -0.25, neutral is -0.25 to 0.25, and positive is 0.25 to 1.0.
Magnitude is from 0.0 to +inf. The higher the magnitude the stronger the positive or negative sentiment, based on the score. Lower magnitude indicates statements balancing others out, while higher values indicate the weight of the score overall. Because the Magnitude can extend to infinity it seems like there is far too much room to wonder about its value compared to scores for other blocks of text. Very subjective.
| Sample #1 – LinkedIn Bio | Sample #2 – The Raven | Sample #3 – Editorial | Sample #4 – Dog Tweet |
| IT professional Salience: 0.2407715 Sentiment: 0.1 [magnitude] 0 [score] | chamber door Salience: 0.33098266 Sentiment: 0.3 [magnitude] 0 [score] | cryptocurrencies Salience: 0.036509313 Sentiment: 0.9 [magnitude] 0.9 [score] | Meet Yogi Salience: 0.8369134 Sentiment: 0 [magnitude] 0 [score] |
| cloud Salience: 0.098823205 Sentiment: 0.2 [magnitude] 0.2 [score] | Lenore Nameless Salience: 0.067261524 Sentiment: 2 [magnitude] -0.2 [score] | one Salience: 0.03649575 Sentiment: 0.5 [magnitude] 0.5 [score] | trip Salience: 0.07094695 Sentiment: 0 [magnitude] 0 [score] |
| manager Salience: 0.0476862 Sentiment: 0 [magnitude] 0 [score] | ember Salience: 0.024451666 Sentiment: 1.2 [magnitude] -0.6 [score] | program Salience: 0.033484023 Sentiment: 0.1 [magnitude] 0 [score] | beach Salience: 0.048720725 Sentiment: 0 [magnitude] 0 [score] |
| role Salience: 0.041792165 Sentiment: 0 [magnitude] 0 [score] | tapping Salience: 0.022468273 Sentiment: 0 [magnitude] 0 [score] | kittens Salience: 0.030432394 Sentiment: 0.9 [magnitude] -0.9 [score] | sand Salience: 0.043418925 Sentiment: 0.8 [magnitude] -0.8 [score] |
| responsibilities Salience: 0.036698595 Sentiment: 0.4 [magnitude] 0.2 [score] | lore Salience: 0.02031062 Sentiment: 0 [magnitude] 0 [score] | internet Salience: 0.024274383 Sentiment: 0 [magnitude] 0 [score] | |
| contributor Salience: 0.033336643 Sentiment: 0 [magnitude] 0 [score] | volume Salience: 0.02031062 Sentiment: 0.9 [magnitude] -0.9 [score] | contracts Salience: 0.018840473 Sentiment: 0 [magnitude] 0 [score] | |
| organization Salience: 0.03316731 Sentiment: 0.1 [magnitude] 0.1 [score] | heart Salience: 0.016615199 Sentiment: 0.8 [magnitude] -0.4 [score] | cryptocurrency Salience: 0.017475905 Sentiment: 0 [magnitude] 0 [score] | |
| solution Salience: 0.026711168 Sentiment: 0.7 [magnitude] -0.3 [score] | more Salience: 0.015859045 Sentiment: 0.1 [magnitude] -0.1 [score] | world computer Salience: 0.017475905 Sentiment: 0 [magnitude] 0 [score] | |
| load balancing solution Salience: 0.019928368 Sentiment: 0.2 [magnitude] 0.1 [score] | nothing Salience: 0.015256954 Sentiment: 0.2 [magnitude] -0.2 [score] | cryptocurrencies Salience: 0.01685153 Sentiment: 0.2 [magnitude] -0.1 [score] | |
| Program Management Salience: 0.016665569 Sentiment: 0.3 [magnitude] 0.3 [score] | word Salience: 0.015148128 Sentiment: 0.8 [magnitude] -0.4 [score] | plague Salience: 0.016228218 Sentiment: 0.7 [magnitude] -0.7 [score] | |
| manager/director role Salience: 0.015977792 Sentiment: 0 [magnitude] 0 [score] | human being Salience: 0.014992289 Sentiment: 0.5 [magnitude] 0 [score] | game Salience: 0.014894219 Sentiment: 0 [magnitude] 0 [score] | |
| company Salience: 0.014537535 Sentiment: 0.9 [magnitude] 0.9 [score] | visitor Salience: 0.012147822 Sentiment: 0.4 [magnitude] -0.4 [score] | banking system Salience: 0.014547589 Sentiment: 0 [magnitude] 0 [score] | |
| expertise Salience: 0.0144847995 Sentiment: 0.1 [magnitude] 0.1 [score] | master Salience: 0.011745481 Sentiment: 1.2 [magnitude] -0.2 [score] | system Salience: 0.014547589 Sentiment: 0.1 [magnitude] -0.1 [score] | |
| roles Salience: 0.013508974 Sentiment: 0.2 [magnitude] 0.2 [score] | Raven “Nevermore Salience: 0.011127895 Sentiment: 2.4 [magnitude] -0.2 [score] | computer code Salience: 0.014547589 Sentiment: 0 [magnitude] 0 [score] | |
| goal Salience: 0.012724441 Sentiment: 0.9 [magnitude] 0.9 [score] | floor Salience: 0.010994431 Sentiment: 0.1 [magnitude] -0.1 [score] | parts Salience: 0.011588174 Sentiment: 0.8 [magnitude] 0.8 [score] | |
| interests Salience: 0.012678256 Sentiment: 0.2 [magnitude] 0.2 [score] | morrow Salience: 0.010954482 Sentiment: 0 [magnitude] 0 [score] | neither Salience: 0.011522884 Sentiment: 0.4 [magnitude] 0.2 [score] | |
| Web Services Salience: 0.011578682 Sentiment: 0 [magnitude] 0 [score] | ghost Salience: 0.009621214 Sentiment: 0.2 [magnitude] -0.2 [score] | news Salience: 0.0093484325 Sentiment: 0 [magnitude] 0 [score] | |
| servers Salience: 0.011578682 Sentiment: 0.1 [magnitude] 0.1 [score] | maiden Salience: 0.009586242 Sentiment: 0.5 [magnitude] 0.5 [score] | dream Salience: 0.00748501 Sentiment: 0.5 [magnitude] 0.2 [score] | |
| experience Salience: 0.010530363 Sentiment: 0 [magnitude] 0 [score] | books surcease Salience: 0.009586242 Sentiment: 0 [magnitude] 0 [score] | programmers Salience: 0.0069575156 Sentiment: 0 [magnitude] 0 [score] | |
| customers Salience: 0.010236813 Sentiment: 0 [magnitude] 0 [score] | sorrowsorrow Salience: 0.009586242 Sentiment: 0 [magnitude] 0 [score] | need Salience: 0.0068155993 Sentiment: 0.3 [magnitude] -0.1 [score] |
IBM Cloud (Watson) Natural Language Understanding
Features
IBM’s feature set is the most mature among the services I tested. This shouldn’t be a surprise considering their success with Watson over the years and the training they’ve been able to apply to their artificial intelligence platform. This article only covers the sentiment and emotion functions.
Categories
Categorize your content using a five-level classification hierarchy. View the complete list of categories here.
Concepts
Identify high-level concepts that aren’t necessarily directly referenced in the text
Emotion
Analyze emotion conveyed by specific target phrases or by the document as a whole. You can also enable emotion analysis for entities and keywords that are automatically detected by the service.
Entities
Find people, places, events, and other types of entities mentioned in your content. View the complete list of entity types and subtypes here.
Keywords
Search your content for relevant keywords.
Metadata
For HTML and URL input, get the author of the webpage, the page title, and the publication date.
Relations
Recognize when two entities are related, and identify the type of relation.
Semantic Roles
Parse sentences into subject-action-object form, and identify entities and keywords that are subjects or objects of an action.
Sentiment
Analyze the sentiment toward specific target phrases and the sentiment of the document as a whole. You can also get sentiment information for detected entities and keywords by enabling the sentiment option for those features. As you’ll read below this maturity and ease of use applies to their API as well.
Pricing
Pricing depends on your overall IBM Cloud subscription plan. On the Lite (read free) plan you get up to 30,000 NLU items per month. Whereas if you upgrade to the paid Standard plan you are purely usage based on every item request, with discounts based on volume. I found it interesting that when you sign up for the Lite plan they don’t even ask for a credit card number.
An NLU item is based on the number of data units enriched and the number of enrichment features applied. A data unit is 10,000 characters or less. For example: extracting Entities and Sentiment from 15,000 characters of text is (2 Data Units * 2 Enrichment Features) = 4 NLU Items.
Limits
This was incredibly difficult to track down on the IBM website. Based on 30 July 2017 Release Notes, text greater than 50K chars will be truncated, the previous limit was 1 kilobytes (1024 bytes). Fifty thousand characters is considerably higher than the other services discussed in this post.
API
The IBM Cloud website only details their REST API so that’s what I used to code against. However, an after the fact search revealed there is a PHP SDK (https://github.com/CognitiveBuild/WatsonPHPSDK or https://github.com/ThomasIBM/php-sdk).
A nice plus for IBM’s API is that you can request multiple features in each API call. They still charge you per feature requested but it’s a lot more convenient to be able to ask once for everything you want rather than making individual calls for each function.
<?php
// IBM will give you a specific username and password to access the NLU service
$ibm_user = '';
$ibm_pass = '';
$ibm_endpoint = 'https://gateway.watsonplatform.net/natural-language-understanding/api/v1/analyze?version=2017-02-27';
// The text you want to analyze
$text = '';
// Do the work and get your result
$data = array(
'text' => "$text",
'features' => array(
'keywords' => array(
'sentiment' => true,
'emotion' => true,
)
),
);
$data = json_encode($data);
$ibm = curl_init();
curl_setopt($ibm, CURLOPT_URL, $ibm_endpoint);
curl_setopt($ibm, CURLOPT_POST, 1);
curl_setopt($ibm, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ibm, CURLOPT_HTTPHEADER, array(
'Content-Type: application/json',
'Accept: application/json'
));
curl_setopt($ibm, CURLOPT_USERPWD, "$ibm_user:$ibm_pass");
curl_setopt($ibm, CURLOPT_HTTPAUTH, CURLAUTH_BASIC);
curl_setopt($ibm, CURLOPT_POSTFIELDS, $data);
$response = curl_exec($ibm);
$ibm_result = json_decode($response, true);
curl_close($ibm);
// $ibm_result will be an array you can access for the returned data
echo 'IBM Cloud / Watson' . PHP_EOL;
echo 'Keyphrases: ' . PHP_EOL;
foreach ( $ibm_result['keywords'] as $i ) {
echo $i['text'] . PHP_EOL;
echo 'Relevance: ' . $i['relevance'] . PHP_EOL;
echo 'Sentiment: ' . $i['sentiment']['score'] . PHP_EOL;
echo 'Emotions: ' . PHP_EOL;
echo ' -Sadness: ' . $i['emotion']['sadness'] . PHP_EOL;
echo ' -Joy: ' . $i['emotion']['joy'] . PHP_EOL;
echo ' -Fear: ' . $i['emotion']['fear'] . PHP_EOL;
echo ' -Disgust: ' . $i['emotion']['disgust'] . PHP_EOL;
echo ' -Anger: ' . $i['emotion']['anger'] . PHP_EOL;
echo PHP_EOL;
}
// Alternately, if you'd like to see the full set of data returned
print_r($ibm_result);
?>
Results
Based on the maturity of Watson, perhaps I shouldn’t be surprised that I found its results the most relevant and accurate.
Watson returned tight and accurate keywords and phrases and caught most of what I felt were important. I was very happy with the results. I think Watson was the only engine to specifically pick out “Raven” for Sample #2. The relevance scores were pretty good but I found the sentiment and emotion values to be the most interesting in this context of keywords rather than the entire block of text. Perhaps it works better with dramatic text found in short stories and poems but to see the word “raven” get a relatively high level of certainty for Sadness, Joy, Fear, and Anger (leaving out only Disgust) seemed very appropriate.
Sentiment score ranging from -1 (negative sentiment) to 1 (positive sentiment). Emotion scores ranging from 0 to 1 for sadness, joy, fear, disgust, and anger. A 0 means the text doesn’t convey the emotion, and a 1 means the text definitely carries the emotion.
| Sample #1 – LinkedIn Bio | Sample #2 – The Raven | Sample #3 – Editorial | Sample #4 – Dog Tweet |
| cloud platform APIs Relevance: 0.995924 Sentiment: 0 Emotions: -Sadness: 0.055568 -Joy: 0.211603 -Fear: 0.046425 -Disgust: 0.013121 -Anger: 0.02802 | chamber door Relevance: 0.901334 Sentiment: -0.0842639 Emotions: -Sadness: 0.122429 -Joy: 0.629781 -Fear: 0.202819 -Disgust: 0.152286 -Anger: 0.245196 | real currency Relevance: 0.900134 Sentiment: -0.486542 Emotions: -Sadness: 0.06918 -Joy: 0.263669 -Fear: 0.086167 -Disgust: 0.036775 -Anger: 0.123255 | sand Relevance: 0.988505 Sentiment: -0.672347 Emotions: -Sadness: 0.601929 -Joy: 0.047119 -Fear: 0.177971 -Disgust: 0.35354 -Anger: 0.046075 |
| Amazon Web Services Relevance: 0.946893 Sentiment: 0.32613 Emotions: -Sadness: 0.088253 -Joy: 0.082979 -Fear: 0.008404 -Disgust: 0.020818 -Anger: 0.0714 | lost Lenore Relevance: 0.501634 Sentiment: -0.568457 Emotions: -Sadness: 0.764473 -Joy: 0.008086 -Fear: 0.247296 -Disgust: 0.087756 -Anger: 0.197887 | Beanie Baby craze Relevance: 0.891312 Sentiment: -0.558869 Emotions: -Sadness: 0.724752 -Joy: 0.124824 -Fear: 0.095344 -Disgust: 0.04535 -Anger: 0.09825 | trip Relevance: 0.845717 Sentiment: 0 Emotions: -Sadness: 0.082996 -Joy: 0.768652 -Fear: 0.055204 -Disgust: 0.02813 -Anger: 0.057443 |
| technical project management Relevance: 0.918105 Sentiment: 0.922231 Emotions: -Sadness: 0.239939 -Joy: 0.364821 -Fear: 0.112302 -Disgust: 0.038732 -Anger: 0.208574 | separate dying ember Relevance: 0.482499 Sentiment: -0.494962 Emotions: -Sadness: 0.727678 -Joy: 0.012482 -Fear: 0.320462 -Disgust: 0.055322 -Anger: 0.080813 | small Canadian company Relevance: 0.875852 Sentiment: 0.53669 Emotions: -Sadness: 0.081233 -Joy: 0.310396 -Fear: 0.007662 -Disgust: 0.533 -Anger: 0.103954 | beach Relevance: 0.836752 Sentiment: 0 Emotions: -Sadness: 0.082996 -Joy: 0.768652 -Fear: 0.055204 -Disgust: 0.02813 -Anger: 0.057443 |
| page load times Relevance: 0.914624 Sentiment: 0.835434 Emotions: -Sadness: 0.108787 -Joy: 0.448846 -Fear: 0.035503 -Disgust: 0.004488 -Anger: 0.025469 | meaninglittle relevancy bore Relevance: 0.459712 Sentiment: -0.825676 Emotions: -Sadness: 0.337678 -Joy: 0.154829 -Fear: 0.321075 -Disgust: 0.050412 -Anger: 0.311102 | suitably malicious webpage Relevance: 0.865377 Sentiment: -0.781828 Emotions: -Sadness: 0.494608 -Joy: 0.107754 -Fear: 0.086254 -Disgust: 0.245535 -Anger: 0.212926 | |
| agents increasing service Relevance: 0.9146 Sentiment: 0 Emotions: -Sadness: 0.2539 -Joy: 0.097912 -Fear: 0.065398 -Disgust: 0.13268 -Anger: 0.096632 | lamp-light gloating o’er Relevance: 0.445707 Sentiment: 0 Emotions: -Sadness: 0.188522 -Joy: 0.186324 -Fear: 0.182589 -Disgust: 0.200685 -Anger: 0.047279 | Ethereum Relevance: 0.747639 Sentiment: -0.513226 Emotions: -Sadness: 0.16149 -Joy: 0.191175 -Fear: 0.080751 -Disgust: 0.055043 -Anger: 0.517517 | |
| complex multi-million-dollar solutions Relevance: 0.912594 Sentiment: 0.922231 Emotions: -Sadness: 0.239939 -Joy: 0.364821 -Fear: 0.112302 -Disgust: 0.038732 -Anger: 0.208574 | ominous bird Relevance: 0.442842 Sentiment: 0 Emotions: -Sadness: 0.21515 -Joy: 0.089418 -Fear: 0.196796 -Disgust: 0.15498 -Anger: 0.209489 | legitimate banking Relevance: 0.744846 Sentiment: 0 Emotions: -Sadness: 0.205808 -Joy: 0.159109 -Fear: 0.049965 -Disgust: 0.013543 -Anger: 0.14675 | |
| HPE Helion Relevance: 0.798603 Sentiment: 0 Emotions: -Sadness: 0.154141 -Joy: 0.172429 -Fear: 0.095666 -Disgust: 0.076074 -Anger: 0.094764 | Raven Relevance: 0.440981 Sentiment: 0.424863 Emotions: -Sadness: 0.694532 -Joy: 0.690868 -Fear: 0.687529 -Disgust: 0.0456 -Anger: 0.680811 | fashionable cryptocurrencies Relevance: 0.73987 Sentiment: -0.420752 Emotions: -Sadness: 0.204408 -Joy: 0.413299 -Fear: 0.058367 -Disgust: 0.092789 -Anger: 0.086215 | |
| cloud solutions Relevance: 0.791795 Sentiment: 0.32613 Emotions: -Sadness: 0.088253 -Joy: 0.082979 -Fear: 0.008404 -Disgust: 0.020818 -Anger: 0.0714 | Nevermore Relevance: 0.42179 Sentiment: 0.436053 Emotions: -Sadness: 0 -Joy: 0 -Fear: 0 -Disgust: 0 -Anger: 0 | imaginary kitten Relevance: 0.728115 Sentiment: -0.387433 Emotions: -Sadness: 0.201491 -Joy: 0.487955 -Fear: 0.101491 -Disgust: 0.053981 -Anger: 0.071164 | |
| Google Adwords Relevance: 0.764266 Sentiment: 0 Emotions: -Sadness: 0.191606 -Joy: 0.134409 -Fear: 0.037958 -Disgust: 0.051213 -Anger: 0.064268 | gently rapping Relevance: 0.418179 Sentiment: 0 Emotions: -Sadness: 0.261638 -Joy: 0.01624 -Fear: 0.253446 -Disgust: 0.49803 -Anger: 0.30956 | imaginary kittens Relevance: 0.725727 Sentiment: 0.53669 Emotions: -Sadness: 0.081233 -Joy: 0.310396 -Fear: 0.007662 -Disgust: 0.533 -Anger: 0.103954 | |
| Microsoft Azure Relevance: 0.75976 Sentiment: 0.32613 Emotions: -Sadness: 0.088253 -Joy: 0.082979 -Fear: 0.008404 -Disgust: 0.020818 -Anger: 0.0714 | midnight dreary Relevance: 0.4073 Sentiment: -0.722491 Emotions: -Sadness: 0.079813 -Joy: 0.253367 -Fear: 0.061463 -Disgust: 0.114973 -Anger: 0.038623 | large parts Relevance: 0.725197 Sentiment: 0 Emotions: -Sadness: 0.205808 -Joy: 0.159109 -Fear: 0.049965 -Disgust: 0.013543 -Anger: 0.14675 | |
| anti-fraud solution Relevance: 0.756846 Sentiment: 0 Emotions: -Sadness: 0.279138 -Joy: 0.240411 -Fear: 0.111477 -Disgust: 0.118215 -Anger: 0.331585 | stately Raven Relevance: 0.404736 Sentiment: -0.368547 Emotions: -Sadness: 0.203589 -Joy: 0.334872 -Fear: 0.14134 -Disgust: 0.04234 -Anger: 0.071284 | gold vanish Relevance: 0.711218 Sentiment: -0.687961 Emotions: -Sadness: 0.506992 -Joy: 0.158692 -Fear: 0.133844 -Disgust: 0.156028 -Anger: 0.210273 | |
| Google Analytics Relevance: 0.749963 Sentiment: 0 Emotions: -Sadness: 0.163427 -Joy: 0.163105 -Fear: 0.063402 -Disgust: 0.063864 -Anger: 0.079907 | curious volume Relevance: 0.404517 Sentiment: -0.365095 Emotions: -Sadness: 0.27674 -Joy: 0.138882 -Fear: 0.145411 -Disgust: 0.033914 -Anger: 0.126784 | tradeable value Relevance: 0.710171 Sentiment: -0.585778 Emotions: -Sadness: 0.312596 -Joy: 0.257172 -Fear: 0.098399 -Disgust: 0.010844 -Anger: 0.308837 | |
| cloud-focused company Relevance: 0.749223 Sentiment: 0.713568 Emotions: -Sadness: 0.039198 -Joy: 0.585749 -Fear: 0.004257 -Disgust: 0.014587 -Anger: 0.005712 | fantastic terrors Relevance: 0.398956 Sentiment: 0 Emotions: -Sadness: 0.06198 -Joy: 0.8648 -Fear: 0.004159 -Disgust: 0.006427 -Anger: 0.004621 | Spectre flaws Relevance: 0.708683 Sentiment: 0 Emotions: -Sadness: 0.337198 -Joy: 0.366092 -Fear: 0.112892 -Disgust: 0.038348 -Anger: 0.095604 | |
| manager/director role Relevance: 0.746573 Sentiment: 0.713568 Emotions: -Sadness: 0.039198 -Joy: 0.585749 -Fear: 0.004257 -Disgust: 0.014587 -Anger: 0.005712 | thy crest Relevance: 0.39861 Sentiment: 0 Emotions: -Sadness: 0.303813 -Joy: 0.161173 -Fear: 0.234302 -Disgust: 0.051151 -Anger: 0.083632 | latest manifestation Relevance: 0.708348 Sentiment: 0 Emotions: -Sadness: 0.198911 -Joy: 0.348413 -Fear: 0.209789 -Disgust: 0.013768 -Anger: 0.072017 | |
| wide range Relevance: 0.745984 Sentiment: 0.32613 Emotions: -Sadness: 0.088253 -Joy: 0.082979 -Fear: 0.008404 -Disgust: 0.020818 -Anger: 0.0714 | uncertain rustling Relevance: 0.397321 Sentiment: -0.568999 Emotions: -Sadness: 0.117186 -Joy: 0.282085 -Fear: 0.03773 -Disgust: 0.065539 -Anger: 0.079408 | network briefly Relevance: 0.70786 Sentiment: -0.322578 Emotions: -Sadness: 0.185012 -Joy: 0.474092 -Fear: 0.127986 -Disgust: 0.114617 -Anger: 0.171347 | |
| legal compliance Relevance: 0.741952 Sentiment: 0 Emotions: -Sadness: 0.122162 -Joy: 0.18645 -Fear: 0.12973 -Disgust: 0.074024 -Anger: 0.140856 | placid bust Relevance: 0.396545 Sentiment: -0.689534 Emotions: -Sadness: 0.893423 -Joy: 0.003638 -Fear: 0.140634 -Disgust: 0.085322 -Anger: 0.092297 | intense speculation Relevance: 0.705586 Sentiment: 0.551383 Emotions: -Sadness: 0.230166 -Joy: 0.424813 -Fear: 0.062365 -Disgust: 0.023201 -Anger: 0.074579 | |
| web technologies Relevance: 0.739898 Sentiment: 0 Emotions: -Sadness: 0.299532 -Joy: 0.296768 -Fear: 0.054391 -Disgust: 0.034434 -Anger: 0.074903 | sculptured bust Relevance: 0.394975 Sentiment: -0.569724 Emotions: -Sadness: 0.132652 -Joy: 0.083568 -Fear: 0.237606 -Disgust: 0.228994 -Anger: 0.316378 | credit card Relevance: 0.703885 Sentiment: -0.558869 Emotions: -Sadness: 0.724752 -Joy: 0.124824 -Fear: 0.095344 -Disgust: 0.04535 -Anger: 0.09825 | |
| world-class customers Relevance: 0.736487 Sentiment: 0.922231 Emotions: -Sadness: 0.239939 -Joy: 0.364821 -Fear: 0.112302 -Disgust: 0.038732 -Anger: 0.208574 | whispered word Relevance: 0.394947 Sentiment: 0 Emotions: -Sadness: 0.084248 -Joy: 0.310103 -Fear: 0.060499 -Disgust: 0.13849 -Anger: 0.190855 | sober realism Relevance: 0.700107 Sentiment: 0 Emotions: -Sadness: 0.194125 -Joy: 0.324705 -Fear: 0.089495 -Disgust: 0.043107 -Anger: 0.062661 | |
| Stratoscale Symphony Relevance: 0.733062 Sentiment: 0.292848 Emotions: -Sadness: 0.131983 -Joy: 0.191748 -Fear: 0.08108 -Disgust: 0.07167 -Anger: 0.084732 | radiant maiden Relevance: 0.393484 Sentiment: 0.745542 Emotions: -Sadness: 0.098052 -Joy: 0.587861 -Fear: 0.032495 -Disgust: 0.004877 -Anger: 0.023884 | cartoon cats Relevance: 0.699942 Sentiment: 0.53669 Emotions: -Sadness: 0.081233 -Joy: 0.310396 -Fear: 0.007662 -Disgust: 0.533 -Anger: 0.103954 | |
| numerous initiatives Relevance: 0.729417 Sentiment: 0 Emotions: -Sadness: 0.072441 -Joy: 0.10439 -Fear: 0.131526 -Disgust: 0.035598 -Anger: 0.056772 | thy memories Relevance: 0.392245 Sentiment: 0 Emotions: -Sadness: 0.462287 -Joy: 0.351459 -Fear: 0.040361 -Disgust: 0.011831 -Anger: 0.0264 | excellent medium Relevance: 0.699505 Sentiment: 0.53669 Emotions: -Sadness: 0.081233 -Joy: 0.310396 -Fear: 0.007662 -Disgust: 0.533 -Anger: 0.103954 |
I’d love to read your questions and comments on these services. How do you think you can use these in your business or project?