Recognition and Reconstruction of Elements in Scanned Pages

01 · Project Information

• Project Duration: January 2023 - March 2023

• Role: Team Leader

• Advisor: Prof. José Miguel Hernández-Lobato, University of Cambridge

02 · Project Overview

This project focused on developing a robust system for recognizing and reconstructing different elements within scanned pages.

The system was designed for digitized documents containing distortion, low resolution, background noise, and varying page layouts. It combined Optical Character Recognition with deep learning-based layout analysis to extract textual and structural information.

The system recognized page elements including:

• Body paragraphs

• Headings

• Tables

• Images

• Headers and footers

• Footnotes

• Captions

• Multi-column text

After recognizing the text and page components, the system reconstructed the logical relationships between the elements to recover the document structure.

03 · Project Background

Traditional OCR systems can process clean and regularly formatted printed documents, but their performance often decreases when handling historical records, damaged pages, low-resolution scans, rotated text, or complex multi-column layouts.

Document digitization requires more than recognizing individual characters. A complete system must also determine:

• Which text regions belong to the same paragraph

• Whether an element is a heading, paragraph, or footnote

• Where tables and images are located

• Whether the page uses a multi-column layout

• What the intended reading order is

• How the logical document structure can be restored

This project combined text detection, text recognition, object detection, instance segmentation, and layout analysis into a unified document-processing pipeline.

04 · Project Objectives

The primary objectives were to:

1. Detect and recognize text from scanned pages

2. Improve robustness against noise, low resolution, and image distortion

3. Identify paragraphs, headings, tables, images, and footnotes

4. Analyze multi-column pages and complex layouts

5. Reconstruct the logical structure and reading order

6. Support multiple languages and document types

7. Develop a scalable approach for large-scale document digitization

05 · System Architecture

The system consisted of the following modules:

• Scanned-image input

• Image preprocessing

• Text-region detection

• Optical Character Recognition

• Page-element detection and segmentation

• Element classification

• Page-layout analysis

• Document-structure reconstruction

• Output generation and evaluation

The complete workflow was:

1. Input a scanned page

2. Preprocess the image

3. Detect text regions using DBNet

4. Recognize text using PaddleOCR

5. Detect and segment page elements using Mask R-CNN

6. Analyze page structure using LayoutParser

7. Classify the type and position of each element

8. Recover the page reading order

9. Reconstruct the logical document structure

10. Export textual and structured results

06 · Technical Implementation

1. Image Preprocessing

Scanned pages may contain rotation, blur, background noise, uneven lighting, and low resolution.

The system therefore applied preprocessing before text recognition and layout analysis.

The preprocessing operations included:

• Image resizing

• Grayscale conversion

• Contrast enhancement

• Noise reduction

• Page-skew correction

• Orientation adjustment

• Image binarization

• Low-quality region detection

The processed images were passed to the OCR and layout-analysis modules.

2. Text Detection with DBNet

The project used PaddleOCR for the OCR pipeline and DBNet to detect text regions within scanned pages.

DBNet uses differentiable binarization to separate text regions from complex backgrounds. It is suitable for irregular, rotated, or curved text.

DBNet was selected because of its ability to:

• Detect text with complex shapes

• Process rotated and curved text

• Remain robust against background noise

• Handle different text scales

• Balance detection accuracy and computational efficiency

The model was trained and fine-tuned using a dataset containing scanned documents, handwritten text, and printed materials from different sources.

3. Text Recognition with PaddleOCR

After DBNet identified the text regions, PaddleOCR recognized the textual content within each region.

The recognition module was designed to process:

• Different fonts

• Different text sizes

• Handwritten and printed text

• Multilingual content

• Low-resolution characters

• Rotated text

• Damaged or partially missing text

On a validation dataset containing 10,000 scanned document images, the OCR pipeline achieved:

• 96% precision

• 94% recall

The system maintained reliable extraction performance on low-resolution and distorted images.

4. Page-Element Recognition with Mask R-CNN

The project used Mask R-CNN to detect and segment different types of page elements.

In addition to predicting bounding boxes, Mask R-CNN generates pixel-level segmentation masks. This allowed the system to distinguish neighboring or irregularly shaped page regions.

The main categories included:

• Headings

• Paragraphs

• Tables

• Images

• Headers

• Footers

• Footnotes

• Captions

The model was trained using 1,500 multi-format documents, including books, articles, reports, and newspapers. It achieved 95% accuracy in page-element classification.

5. Layout Analysis with LayoutParser

Mask R-CNN was integrated with the LayoutParser toolkit to perform fine-grained document-structure analysis.

LayoutParser was used to:

• Manage page-element detection results

• Analyze spatial relationships between elements

• Distinguish paragraphs, footnotes, and captions

• Identify multi-column layouts

• Determine page regions

• Recover reading order

• Produce structured page representations

Automated document segmentation reduced the need for manual preprocessing.

6. Document-Structure Reconstruction

After text recognition and element classification, the system reconstructed the logical organization of the scanned page.

The reconstruction process considered:

• The position of each element

• Spatial relationships between regions

• Hierarchical relationships between headings and paragraphs

• Associations between tables, images, and captions

• Separation of headers and footers from body content

• Reading order in multi-column layouts

• Continuity between related regions

The system combined OCR results with layout information to produce a structured document containing text, coordinates, element categories, and reading order.

07 · Dataset

The system was tested using a dataset containing 10,000 scanned images.

The dataset included:

• Distorted scanned images

• Handwritten text

• Printed text

• Low-resolution pages

• Historical documents

• Damaged pages

• Multilingual content

• Single-column and multi-column layouts

• Books, articles, reports, and newspapers

The diversity of document types and image conditions supported the evaluation of system generalizability.

An additional dataset of 1,500 multi-format documents was used to train the layout-analysis model.

08 · Performance Evaluation

The system was evaluated in terms of text detection, text recognition, layout classification, and overall processing accuracy.

The final results included:

• 96% precision for the DBNet-based OCR pipeline

• 94% recall for text recognition

• 95% accuracy for page-element classification

• 97% overall system accuracy

• Evaluation across 10,000 scanned images

• Robust processing of low-quality and distorted pages

• Successful extraction from historical and damaged documents

• Support for multilingual content and diverse layouts

09 · Challenges and Solutions

10 · My Responsibilities

As the team leader, I was responsible for:

• Defining project objectives and the implementation plan

• Designing the overall system architecture

• Dividing the project into OCR and layout-analysis tasks

• Coordinating team responsibilities and progress

• Participating in the scanned-image preprocessing design

• Supporting the integration of PaddleOCR and DBNet

• Participating in the Mask R-CNN layout-analysis module

• Integrating LayoutParser with detection results

• Designing system testing and performance evaluation

• Analyzing recognition performance on low-quality documents

• Organizing system integration, results, and final presentation

11 · Key Contributions and Outcomes

• Developed a deep learning-based scanned-document recognition system

• Implemented text detection and recognition using PaddleOCR and DBNet

• Achieved 96% OCR precision

• Achieved 94% OCR recall

• Implemented page-element detection and segmentation using Mask R-CNN

• Achieved 95% page-element classification accuracy

• Integrated LayoutParser for automated layout analysis

• Evaluated the system using 10,000 scanned images

• Achieved 97% overall system accuracy

• Supported low-resolution, distorted, multilingual, and complex-layout documents

• Processed historical and damaged scanned pages

• Designed a scalable architecture for large-scale document digitization

12 · Applications

The system has potential applications in:

• Historical document digitization

• Library and archive management

• Legal document processing

• Financial and commercial document recognition

• Academic-paper digitization

• Newspaper and journal extraction

• Intelligent document retrieval

• Automated information entry

• Accessible document reading

• Cultural heritage preservation

By identifying both text and structural information, the system can reduce manual processing and generate structured data for information retrieval, content analysis, and knowledge management.

13 · Conclusion

This project combined deep learning-based OCR with page-layout analysis to develop a scanned-page element recognition and reconstruction system.

DBNet and PaddleOCR were used for text detection and recognition, while Mask R-CNN and LayoutParser supported page-element classification, instance segmentation, and logical layout analysis.

Through the project, I strengthened my skills in computer vision, OCR, object detection, instance segmentation, and document-layout analysis. I also gained further experience in deep learning model integration, data processing, system evaluation, and team leadership.

14 · Future Work

Future development could focus on:

• Introducing multimodal learning for complex handwriting recognition

• Applying Natural Language Processing for contextual understanding

• Generating document summaries and structured indexes

• Improving table structure and cell-relation recognition

• Recovering cross-page content and reading order

• Compressing models for mobile and edge devices

• Supporting real-time document scanning and processing

• Expanding support for additional languages and historical fonts