The Rise of Generative IDP - AI Meets Document Capture

Intelligent Document Capture (IDP) refers to the use of software to classify documents and extract and validate data. IDP tools are used to process documents like invoices, receipts, forms, contracts, and statements (for use cases like accounts payable, loan processing, compliance checks...). Traditional IDP solutions (such as ABBYY FlexiLayout, OpenText Captiva, IBM Datacap, Kofax/Tungsten, etc.) have delivered significant efficiency gains by reducing manual data entry. But they also have limitations in handling diverse document layouts and unstructured text. LLMs like GPT4, Gemini, or Llama 2 are bringing new capabilities for IDP. These models can understand and generate natural language, offering the potential to improve how documents are processed today. In this article, we’ll look at the challenges faced by IDP tools, explore how GenAI could help address some of these issues and touch on a few technical considerations.  

[Challenges with Traditional IDP] 

[1] Template Maintenance – Tools depend on forms and templates to locate data on image, someone would define zones or patterns (e.g invoice total is in the bottom right corner after the text Total:). If a document layout changed (say a new invoice format from a supplier), the template had to be updated. Systems could not handle new formats without manual reconfiguration or maintenance. If a vendor invoice design changes or a new vendor is added, automation breaks until a new template is built. 

[2] Need for Large Training Datasets – After some advancements around 2010 Machine Learning and Deep Learning were introduced in IDP tools. Rather than rigid templates, systems began training models. Companies had to collect and label hundreds of sample documents for each document type to train the models. This meant long project lead times to define and validate the model. Straight through processing rates improved with ML, but achieving high accuracy rate with inputs document varying remained challenging. 

[3] Difficulty with Unstructured Content - Traditional IDP is good at extracting structured data, but much less so at understanding free-form text. A rules-based system might pull out dates and names from a contract but could not summarize the contract provisions. Meaning contextual understanding was limited. If a financial analyst asks, “what are the key obligations of this loan agreement?” or “does this insurance policy cover floods?” traditional tools could not provide an answer. 

[4] High Exception Rates and Manual Review - Legacy IDP pipelines often had modest straight-through automation rates. They might auto-process a portion of documents, but many cases fell out as exceptions that required human correction.  

[5] Rigid and Narrow AI - Models in IDP are narrow classifiers or field extractors. It couldn’t adapt to new document types without explicit re-training. For example, ML models trained to extract data from invoices wouldn’t magically work on a bank statement – you need a separate project to train. 

Overall traditional IDP tools brought automation, but they were constrained by their need for templates, huge training data and limited comprehension. Good at structured, repetitive docs (like standard forms or consistent invoice formats) but struggle with varying templates. This often left a lot of dark data in finance – unused information in documents.  

[Generative AI Capabilities] 

In this new phase LLMs bring new capabilities to IDP. These models (e.g. GPT-4, Gemini, LLaMA) are pretrained on vast amounts of text from the internet, books, and other sources. As a result, these modes have a broad understanding of language, formats, and even some domain knowledge. Many of these features could be game changer of IDP.  

[1] Zero-Shot Learning - LLMs ability to perform tasks with zero or minimal task-specific training. This means an LLM can classify a document or extract a field without having been explicitly trained on that document layout. This capability was unheard of in IDP – traditionally one had to provide training samples for each document type. Now by prompting the model, we can process unseen document formats.  

[2] Unified Document Types & Reduced Needs of Training Data - Generative AI removes the strict division between structured, semi-structured, and unstructured documents. Also, it reduces the need for a collection of large training sets.  Since an LLM treats the input as text sequence, any document that can be turned into text becomes fair input for the LLMs. The model doesn’t care if the text came from a formatted form or a free-flowing letter, it reads and understands the content in either case. GPT-style models can parse all of them and extract meaning, whereas older systems require separate logic for each format. 

[3] Language Understanding - Unlike a template model, LLMs understand natural language. They know that an invoice number is likely a specific sequence of characters often preceded by words like Invoice # or Inv. No. Therefore, if we ask an LLM a question like “What is the invoice number on this document?” or “When does the lease contract expire?”, the model can understand what we’re looking for and check the provided text for answers. This ability to extract information by understanding its meaning is a significant leap. 

[4] Q&A and Summarization - You could feed a lengthy policy document into GPT and get back a summary of the key points (coverage limits, exclusions, etc.). Financial analysts can ask questions in natural language to an LLM about a document or a set of documents.  

[5] Adaptability and Extensibility - Once you have an LLM in your document pipeline, you can repurpose it for many task - classification, extraction, summarization, translation etc. This is different from traditional systems where each new task often required a new module or model. This multitasking ability offers better document understanding, turning documents from static records into actionable insights. 

In essence, generative AI brings greater flexibility, contextual intelligence, and user-friendly interaction to document processing. Instead of coding where to find data, we can ask an AI model in natural language to find it. Also instead of just capturing data, we can derive insights (summaries, answers) from documents. However, these advancements come with new challenges and considerations. Let's go through some technical considerations.  

[Technical Considerations] 

[1] OCR and Data Input - If documents are in image form (scans, photos, faxes), a high-quality OCR is still required to convert them to text for the LLM. Choosing an OCR tool that handles multi-language, mixed printed/handwritten text (if needed), etc., remains important. 

[2] Prompt Design - The real trick to making LLMs work for IDP is all about how you talk to them- prompt engineering, where you craft precise instructions to get exactly what you need. Think of it like giving clear directions to a smart assistant. This is less like traditional coding and more about guiding a powerful AI. 

[3] Model Selection (Cloud vs OnPrem) - Organizations today are trying to decide whether to use public cloud LLMs or to run these models on their own (on-premises). Cloud options offer easy access to LLMs, but there are concerns about data privacy (sending PII data) and ongoing token-based costs. While cloud providers offer assurances about data usage, highly regulated industries like banking often lean towards self-hosting. A promising middle ground could be fine-tuning smaller models in-house for specific IDP tasks. 

[4] Combining with Rule Systems - Straightforward tasks like verifying calculations or date ranges, traditional rule engines or scripts are often more efficient and reliable than LLMs. For example, verifying a sum of line items equals the total, or checking if a date is within a certain range. An effective IDP and Generative AI solution should combine both approaches in a hybrid pipeline. 

[5] Handling Hallucinations and Errors - Hallucinations are a worry with LLMs. One good way to tackle this is by making the LLM show from which part of the image it got the answer (highlight the text/image area). That way you can easily check if the information is actually supported. If it can't give a reference, it's probably guessing. Another good trick is to ask the LLM the same thing in different ways. Another important consideration could be to keep human in the loop for all critical data validation (like invoice amount, claim amount approvals). 

[6] Regulatory Compliance - Using AI brings regulatory oversight, especially under GDPR in the EU for data privacy and the Federal Reserve's SR 11-7 in the US for model risk. Banks will need to prove their LLMs are explainable, transparent about training data, and thoroughly validated. This is trickier with black box third-party models (regulators are catching up). A common strategy to navigate this is to position the LLM as a human assistant, meaning a person still makes the final decision.

[GenAI Integrations and Industry Adoption] 

The rise of GenAI has led to quick adoption by IDP vendors. By late 2023 more than 70 percent of IDP companies had GenAI functionality in release or in development, and today over 90% of IDP products have already incorporated GenAI in some form. Here are some examples,

Almost every major IDP and RPA provider is embedding GenAI, either through partnerships with OpenAI, launching their own LLMs, or integrating with cloud AI services.  

[Summary] 

Financial institutions will soon be all-in on AI-powered document workflows, much like they embraced OCR years ago. Integrating GPT-style LLMs means teams can now process any document format with minimal setup. This speeds up tasks like invoice processing and automates parts of complex work like analyzing legal documents. Early results show higher straight-through processing. 

Saying that Generative AI couldn't be a standalone IDP solution, it needs to be part of a strong system for document management, validation, and governance. Because financial data is so sensitive and mistakes are expensive (imagine paying 100 times more for an invoice or claim, just because LLM hallucinated). Hybrid approach should work – LLM with established rules and human oversight in IDPs. This strategy could reduce risks like hallucinations and keep accuracy and compliance high. 

This next phase (GenAI inclusion) in IDP will turn it from a simple data extraction tool into a powerful document intelligence system. For the finance industry, which handles a massive volume of documents and demands high precision, this is a game-changer, a leap forward that aligns with digital transformation goals. The path ahead is clear - GenAI is slowly becoming an important part of the IDP. We are in a new era of document automation and intelligence. 

[Reference]

The 4 Waves of IDP: an unstoppable AI tide – IDP industry perspective on generative AI as the 4th wave of document processing, with zero-shot capabilities and rapid adoption. intelligentdocumentprocessing.com