Document Processing: Named Entity Recognition with Azure Services

Named Entities are specific pieces of information extracted from unstructured text, such as the names of people, organizations, locations, dates, monetary values and more. This process of identifying and classifying these key elements is known as Named Entity Recognition (NER). By extracting named entities from these documents, enterprises can automate data entry, improve searchability, enhance compliance tracking, accelerate decision-making and gain valuable insights about the document.  

The process of spotting and categorizing information from documents automatically can be tedious. Imagine if you had a smart assistant that could scan these documents, pick out all the pieces of information, saving hours of manual work. That is what an automated solution can do to using modern technology.  

Consider a financial services company that processes hundreds of loan applications daily. Each application contains customer names, addresses, employer information, income details, and dates. Manually reading through these applications to extract key fields slows down operations and increases the risk of errors. By using Azure’s AI-powered services to automatically extract and identify named entities from the uploaded PDFs or scanned documents. 

Before we go into the details of implementation let’s see field types which Azure NER (under Language API) can extract. This service can identify a wide range of entity categories and subcategories, spanning general entities as well as domain-specific entities (e.g. finance-related identifiers, healthcare terms etc). The tables below summarize a few supported entity categories (subcategories and descriptions are removed to keep it short).

Here are high level steps on how you can get the named entities using Azure AI features.  

[1] Take your image/pdf and get the text results using Azure Document Intelligence Read API. Select OCR/Read model.

[2] Upload your image and analyze the response JSON. Try to get the content tag value which typically gives you the OCR text for the image.

Note – You can also make an API call in your application for more details on how to make an API call see this previous article - https://www.linkedin.com/pulse/azure-layout-model-harnessing-key-value-pairs-doc-vijay-chaudhary-zrtxc/ 

[3] Take the text from content tag (you can also write JSON parsing program to take this value for next steps) 

[4] Create Azure Language AI resources and launch Language Studio. Select Extract Named Entities option.  

[5] Put the text from content tag in sample text window to try the feature and hit run.

[6] You would see responses on UI and JSON response (tabs).  

Note – You should be able to invoke end point for the language API call and parse the response in your application.  

[Summary] 

By combining Azure Document Intelligence and Azure Language Services, teams can automate the extraction of meaningful information from scanned PDFs/images. This approach saves manual hours. Automating the reading and understanding of incoming documents helps businesses to enhance searchability, ensure compliance, integrate key data into repository systems to make faster access to information. Adding post-processing steps, such as organizing the extracted named entities based on document types, can make the solution more powerful. Another possibility is to connect the output to Azure Cognitive Search to build a smart, searchable document database. For domain-specific requirements exploring Custom NER models can also be explored. As Azure AI capabilities continue to grow, businesses have an exciting opportunity to keep enhancing their document workflows and unlock more value from unstructured data.