Text Analytics with Azure Cognitive Services

When it comes to extracting valuable insights from unstructured data, I always fall back on Azure cognitive services. This gives a very strong set of build in AI capabilities to easily analyze text and classify unstructured data seamlessly. The Text Analytics API is a cloud-based service that provides Natural Language Processing (NLP) features for text mining and text analysis, including: sentiment analysis, opinion mining, key phrase extraction, language detection, and named entity recognition. Discover insights in unstructured text using natural language processing. Identify key phrases and entities such as people, places, and organizations to understand common topics and trends. Classify medical terminology using domain-specific, pretrained models. Gain a deeper understanding of customer opinions with sentiment analysis. Evaluate text in a wide range of languages.

In this blog I am going to show you how you can use the Text Analytics API in popular programming languages like python to build a simple text analyzer engine. Lets get started with the code

First step is to install the python package for Azure Cognitive Services in your python environment.

pip install azure.cognitiveservices.language.textanalytics

-- coding: utf-8 --
 import os
 from azure.cognitiveservices.language.textanalytics import TextAnalyticsClient
 from msrest.authentication import CognitiveServicesCredentials

If the pip install ran successfully the above code should execute without issues in a Jupyter notebook.

The next step is the create the Azure cognitive services resource in the Azure portal. This is very straight forward and the steps to create the resources are here. Once the resource is created, retrieve the endpoint and the api keys

Continuing the python notebook. Now define the authentication function as shown below.

def authenticateClient():
     credentials = CognitiveServicesCredentials('your api key here')
     text_analytics_client = TextAnalyticsClient(
         endpoint='https://yourtextanalyticsserice namehere.cognitiveservices.azure.com/', credentials=credentials)
     return text_analytics_client

Alternatively, a better approach is to use Azure Keyvault to get the keys or environment variables. I will be making another blog on how to use Azure key vaults to retrieve sensitive information. For now, below is the code to set and get environment variables.

import os 
# Set environment variables 
os.environ['API_KEY'] = 'KEY'  
# Get environment variables 
mykey = os.getenv('API_KEY')

Now that we have setup the Azure resource, authenticated to it and setup the python environment, we are now ready to explore the functionality and use case. Here is a function that will accept a document and return the sentiment score of the document. The score is from 0 to 1. 0 being the most negative and 1 being the most positive in terms of sentiment analysis.

def sentiment():

client = authenticateClient()
try:
   documents = [
             {"id": "1", "language": "en", 
                 "text": "Thank you for all that you're doing"},
             {"id": "2", "language": "en", "text": "I don't like this book at all."},
            {"id": "3", "language": "es", 
                 "text": "No tengo dinero ni nada que dar…"},
            {"id": "4", "languagez": "it",
                 "text": "L'hotel veneziano era meraviglioso. È un bellissimo pezzo di architettura."},
            {"id": "5", "language": "en",
                 "text": "IBM Intg Bus PVU Lic"}
         ]
response = client.sentiment(documents=documents)
         for document in response.documents:
             print("Document Id: ", document.id, ", Sentiment Score: ",
                   "{:.2f}".format(document.score))
except Exception as err:     
       print("Encountered exception. {}".format(err))
sentiment()            

Output

Document Id: 1 , Sentiment Score: 0.99
Document Id: 2 , Sentiment Score: 0.02
Document Id: 4 , Sentiment Score: 0.16
Document Id: 5 , Sentiment Score: 0.50
Document Id: 3 , Sentiment Score: 0.44