Layout-Aware Region Compression for Transformer Document Understanding

Transformer models for document understanding must process text, tables, stamps, signatures, and layout structures at the page level. However, many document images contain repeated headers, blank margins, boilerplate clauses, and low-information background regions. Treating all regions equally increases memory usage and may reduce scalability for enterprise document processing. This study develops a layout-aware region compression method for document understanding. The proposed DocRegion-Slimmer first segments each page into semantic regions, including title blocks, paragraph zones, tables, key-value fields, signatures, seals, and background. It then compresses low-information regions with shared layout embeddings while preserving high-value regions such as numerical fields, table cells, dates, names, and approval marks. A region consistency loss is introduced to avoid losing cross-region dependencies between extracted entities. The experiments covered 128,000 scanned and digital-born pages from invoices, tax forms, purchase orders, insurance claims, contracts, bank statements, and clinical forms. The dataset contained 42 entity types, 16 table structures, and 3.7 million OCR text boxes. DocRegion-Slimmer reduced the average number of visual-layout tokens per page from 1,024 to 438. On entity extraction, micro-F1 changed only slightly from 91.6% to 91.1%, while memory consumption during inference decreased by 44.8%. For table understanding, cell-level F1 reached 88.4%, outperforming a simple patch-dropping baseline by 5.7 points. In long contract pages, the model reduced missing-field errors for dates and monetary values by 19.2% because key-value regions were preserved rather than compressed uniformly. Batch processing throughput increased from 2,900 to 5,180 pages per hour on one A100 GPU. The results indicate that layout-aware compression can improve the efficiency of transformer document understanding without sacrificing critical information extraction accuracy.

1. Lvov, D., Smirnov, I., Volokha, V., Laushkina, A., & Boukhanovsky, A. (2026). Enhancing Technical Question Answering Quality Through Multimodal Document Segmentation. IEEE Access.
2. Zhang, Z. (2026). A Study on the Impact of Cost Presentation on Decision-making Bias and Cancellation Behavior in Online Subscriptions. Available at SSRN 7349839.
3. Du, Y., Liu, Q., Dong, Y., & Chen, Z. (2026). Cross-Domain Transfer and Few-Shot Recognition of Defect Images in Complex Industrial Settings.
4. Zhang, Z., Wang, J., Li, Z., Wang, Y., & Zheng, J. (2025). Anncoder: A mti-agent-based code generation and optimization model. Symmetry, 17(7), 1087.
5. Singh, A., Shah, M., Sampath, D., & Javis AI Team. (2025). Spatial ModernBERT: Spatial-Aware Transformer for Table and Key-Value Extraction in Financial Documents at Scale. arXiv preprint arXiv:2507.08865.
6. Xiong, W., Guo, Y., & Zeng, Y. (2026). A Study on the Energy Efficiency of MEP Systems and Coordinated Dispatch Mechanisms for Multi-energy Systems in High-density Urban Buildings. Available at SSRN 7120518.
7. Liang, S., Hsu, K. H., & Liu, Z. (2026). Rolling Scheduling and Capacity Balancing for Server Assembly Under Engineering Change Disturbances.
8. Bao, Y., Qiu, Y., & Wang, H. (2026). FLARE: A Real-Time Framework for Detecting Malicious Account Activities at Internet Scale. Available at SSRN 7185659.
9. Trivedi, D., & Agarwal, R. (2026). Improving Optical Character Recognition (OCR) Reliability on Noisy Document Images Using Hybrid Deep Learning and Post-Processing Techniques. Cureus Journals, 3(1).
10. Xu, D., Chen, H., & Gui, H. (2026, March). Unified Online Estimation Method for SOC, SOH, and Power Capacity Considering Safety Boundary Consistency in Battery Management Systems. In 2026 International Conference on Electrical, Control and Information Technology (ECITech) (pp. 133-137). IEEE.
11. Zhao, Z., & Welsch, R. E. (2026). Point-in-Time Financial RAG with Frozen LLMs and Market-Feedback Adaptive Retrieval. arXiv preprint arXiv:2605.31201.
12. Yuan, Y., Xu, T., Yin, J., & Huang, J. (2026). Modeling Loading Anomalies and Identifying Root Causes in Media Consumption Workflows on Large Social Media Platforms. Available at SSRN 7393318.
13. Prajapati, A., & Maurya, S. (2025, October). Table structure recognition: a review of current approaches and future opportunities. In International Conference on Advancing Technology in Engineering and Science (ICATES 2025) (Vol. 2025, pp. 86-91). IET.
14. Yuan, Y., Yin, J., Huang, J., & Xu, T. (2026). Weakly Supervised Anomaly Detection and Privacy Risk Scoring in Community Message Threads.
15. Hong, Z., Liang, S., Xu, T., & Chen, H. (2026). A Study on Failover Verification and Recovery Objective Prediction for Cross-Region Cloud Services.
16. Bao, Y., & Wang, H. (2026). Orion: A High-Throughput, Fault-Tolerant Event Routing Architecture for Hyperscale Data Streams. Fault-Tolerant Event Routing Architecture for Hyperscale Data Streams (January 01, 2026).
17. Giovannini, S., & Marinai, S. (2025, October). A survey on reading order, table of contents, and structure extraction in document analysis. In 2025 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) (pp. 7635-7644). IEEE.
18. Yang, Q., & Du, Y. (2026). Predicting Rollback Risks and Controlling Gradual Rollout Traffic for Online Ranking Feature Deployments.
19. Wu, J., Zhang, J., Wu, D., & Peng, Y. (2026). Visual Transformation Mechanisms for Integrating Digital Cultural Heritage Resources into Junior High School Art Curricula.
20. Shihab, I. F., Akter, S., & Sharma, A. (2026). LLM-Guided Probabilistic Fusion for Label-Efficient Document Layout Analysis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 3574-3583).
21. Huang, J., Xu, T., Yin, J., & Yang, J. (2026). Data Quality Monitoring and Intelligent Scheduling Optimization in Financial Data Workflow Automation. Available at SSRN 7179338.
22. Yao, G., Zheng, J., Wang, Z., Zhang, W., Han, R., Zhao, C., ... & Liu, R. (2026, March). V-pruner: A fast and globally-informed token pruning framework for vision transformer. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 40, No. 40, pp. 34396-34404).
23. Abdalla, M., Kasem, M. S., Mahmoud, M., Yagoub, B., Senussi, M. F., Abdallah, A., ... & Kang, H. S. (2025). Receiptqa: A question-answering dataset for receipt understanding. Mathematics, 13(11), 1760.
24. Gu, X. (2026). Identifying Causal Effects and Analyzing Heterogeneity of User Growth Interventions on Digital Platforms: Evidence from Large-Scale Behavioral Data. Available at SSRN 6809181.
25. Zhao, J., Fan, J., & Li, L. (2026). A Study on an Explainable Causal-Enhanced LLM Agent for Predicting the Forming Quality of Automotive Component Materials.
26. Minouei, M. (2026). Structural Information Extraction from Document Images: Addressing Challenges in Layout Analysis, Table Detection, and Classification (Doctoral dissertation, Rheinland-Pfälzische Technische Universität Kaiserslautern-Landau).
27. You, S. (2026). Verifiable Audit Mechanisms in AI Compliance Automation: Scalability. Available at SSRN 6547458.
28. Jiao, Y., Shi, T., Zhao, B., & Wang, A. (2026). Retrieval-Guided Structured Reasoning and Interpretable Representation Learning for Large-Scale Video–Language Models.
29. Liang, S., Du, Y., Chen, W., & Liu, Z. (2026). Order Allocation and Emergency Transportation Decisions Amid Multi-Source Procurement Disruptions.
30. Giovannini, S., & Marinai, S. (2025, October). A survey on reading order, table of contents, and structure extraction in document analysis. In 2025 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) (pp. 7635-7644). IEEE.