Visual NLP

Visual NLP is a document understanding library from John Snow Labs that combines computer vision, OCR, and NLP to extract structured information from unstructured visual documents. It is designed for organizations that need to process PDFs, scanned images, DICOM files, forms, and natural scene images at scale, including use cases in healthcare, finance, and legal document workflows.

The library integrates with John Snow Labs' broader NLP ecosystem and is built to run on distributed clusters, supporting fast inference with hardware optimization. It is a production-grade, licensed product with trial periods available.

Core Capabilities

• Text detection and extraction from images, PDFs, and scanned documents
• Layout analysis to identify document structure
• Visual Named Entity Recognition (Visual NER) for locating and extracting specific facts and figures from custom images and forms
• Visual Document Classification
• Visual Question Answering
• Table detection and extraction, converting rows and columns from scans into structured data frames
• De-identification of images, PDFs, and DICOM files
• DICOM processing, including text recognition from both image content and metadata
• Signature and date detection in image-based forms
• Text recognition in natural scenes using image segmentation and preprocessing techniques

Document Preprocessing Features

• Skewness correction for scanned documents to improve OCR accuracy
• Background noise removal from scanned documents
• Image quality preprocessing that can be fine-tuned for specific OCR requirements
• Spelling correction applied after OCR to improve downstream NLP results

NLP Pipeline Integration

• Supports end-to-end NER pipelines starting from scanned image import through preprocessing, OCR, spell correction, and entity extraction
• Imports scanned images from cloud storage as part of pipeline workflows
• Delivers visually enriched versions of standard NLP tasks, including Visual NER, Visual Document Classification, and Visual Question Answering
• Integrates with all other John Snow Labs NLP products

Trainable and Customizable Models

• Supports training custom models to learn the location, surrounding context, and formatting of specific facts within images and forms
• Allows fine-tuning of image preprocessing steps to optimize OCR results for specific document types

Visual NLP is designed to scale to a cluster and is described as a production-grade, secure product. Input document quality is noted as a factor that can affect output accuracy, particularly for extremely distorted or damaged documents. The tool has been applied in use cases including automated invoice classification, patient record extraction, and large-scale document understanding workflows.