Hybrid PII Detection System

Technical Architecture

PII Regulations by Country

The system is designed to comply with specific PII regulations across LATAM and Malaysia. Below are the key legal frameworks and personal identifier formats detected by the system.

OCR Pipeline with Tesseract

Due to authorization constraints on using modern cloud-based OCR services, the system implements Tesseract OCR for text extraction from images and scanned documents. This choice required significant optimization to achieve acceptable accuracy.

• Image Preprocessing: Adaptive thresholding, noise reduction, and contrast enhancement to improve Tesseract's recognition accuracy on challenging documents
• Multi-language Support: Configured Tesseract for Spanish, Portuguese, English, and Malay to handle all target markets
• Layout Analysis: Custom post-processing to maintain document structure and spatial relationships between text elements
• Confidence Filtering: OCR quality thresholds to reject low-confidence extractions that could lead to false PII positives

OCR Technology Comparison

Transformer Ensemble Detection

Instead of using a single large language model, the system implements an ensemble of specialized transformer models, each fine-tuned for specific PII detection tasks. This approach provides better accuracy, faster inference, and more interpretable results than monolithic LLMs.

• Named Entity Recognition (NER) Models: BERT-based models fine-tuned on Spanish and Portuguese PII datasets to detect names, organizations, and locations
• Document Classification Models: RoBERTa models trained to identify document types (ID cards, contracts, medical records) for targeted PII scanning
• Contextual Understanding Models: XLM-RoBERTa for cross-lingual PII detection in multilingual documents
• Ensemble Voting: Weighted voting mechanism combines predictions from multiple models, with confidence thresholds calibrated per PII type
• Fusion Algorithm: Combines regex pattern matching with transformer ensemble outputs using a custom confidence scoring system to minimize false positives while ensuring high recall
• Continuous Learning: Feedback loop from human-in-the-loop validation improves model performance over time through active learning

Detection Pipeline Integration

The complete detection pipeline integrates OCR, regex patterns, and transformer models in a coordinated workflow:

• Input Processing: Documents are classified (text, image, PDF) and routed through appropriate preprocessing
• OCR Extraction: Tesseract extracts text from images/scans with quality filtering
• Parallel Detection: Regex patterns and transformer ensemble analyze text simultaneously
• Result Fusion: Confidence scores from all detectors are combined using weighted averaging
• Threshold-based Alerting: Configurable confidence thresholds determine when to trigger alerts
• Country-specific Rules: Detection logic adapts to jurisdiction-specific PII requirements

Key Learnings

Building this hybrid PII detection system provided valuable insights into managing compliance across multiple jurisdictions:

• Regulatory Complexity: Each country's PII regulations have unique requirements and identifier formats, requiring a modular architecture that can easily accommodate new jurisdictions
• Ensemble Model Architecture: Combining multiple specialized transformer models with regex provides better accuracy, faster inference, and more interpretable results than monolithic approaches. The modular design allows for targeted improvements without retraining entire systems
• OCR Trade-offs: Authorization constraints forced Tesseract adoption, which required significant preprocessing and confidence filtering. While SOTA OCR or VLMs would provide superior results, the ensemble transformer models compensate by providing strong semantic understanding of the OCR output
• False Positive Management: In compliance scenarios, minimizing false positives is as important as maximizing recall. Multi-level confidence thresholds (OCR quality, regex match strength, transformer confidence) must be carefully calibrated based on business risk tolerance
• Multi-language Challenges: Spanish, Portuguese, English, and Malay have distinct linguistic patterns. XLM-RoBERTa's cross-lingual capabilities combined with language-specific fine-tuned models provide robust multilingual PII detection
• Alert Fatigue: User experience design is critical—too many alerts lead to alert fatigue and reduced compliance. Smart alerting with context, severity levels, and actionable guidance improves adoption and response rates
• On-Premise vs. Cloud: Data privacy and compliance requirements mandated on-premise deployment, limiting access to cloud-based OCR and VLM services. This constraint drove the design of a comprehensive preprocessing and ensemble architecture