Integrate PaddleOCR as an alternative OCR engine

[hwf1324]

Is your feature request related to a problem? Please describe.

PaddleOCR is probably the best open-source traditional OCR out there right now. It works way better than Windows OCR. By switching between different models, it can recognize over 80 languages. You can even use custom models to make it even better.

Describe the solution you'd like

Use WinML to run the PaddleOCR model converted to ONNX.

Currently, there are no open source projects related to WinML inference deployment, so you may need to write the relevant code yourself.

Related: #18662

Describe alternatives you've considered

In the future, when we introduce ONNXRuntime, we can also consider using ONNXRuntime directly.

Currently, there are some PaddleOCR projects that use ONNXRuntime, which may be directly integrated.

Additional context

[josephsl]

Hi, would this be considered a good use case for a pluggable content recognition framework I described a while ago Can we get a proof of concept add-on demonstrating the superiority of this model versus Windows UWP OCR? Thanks.

[hwf1324]

Hi, would this be considered a good use case for a pluggable content recognition framework I described a while ago Can we get a proof of concept add-on demonstrating the superiority of this model versus Windows UWP OCR? Thanks.

In fact, there is currently an add-on (xyOCR) in the add-on store that uses PaddleOCR.

However, this add-on has some issues:

• The bound PaddlePaddle runtime is too large and only supports CPU.
• There are few bound models, and if other language support is required, it needs to be handled manually.

If you want to test the recognition accuracy of Korean, you may need to use other projects such as RapidOCR.

[ABuffEr]

Hi,
very interesting proposal.
I'd suggest also to consider the creation of a multi-engine OCR handler on NVDA side, providing the ability to select the prefered engine installed separately (e.g. via add-on). An option "auto" based over each engine features would be good too (but a bit more complex, I suppose).

[cary-rowen]

Integration of advanced PaddleOCR into NVDA provides text extraction features for mathematical formulas, tables, and complex layout documents.

Quote from PaddleOCR doc:

• PaddleOCR-VL - Multilingual Document Parsing via a 0.9B VLM
  The SOTA and resource-efficient model tailored for document parsing, that supports 109 languages and excels in recognizing complex elements (e.g., text, tables, formulas, and charts), while maintaining minimal resource consumption.

• PP-OCRv5 — Universal Scene Text Recognition
  Single model supports five text types (Simplified Chinese, Traditional Chinese, English, Japanese, and Pinyin) with 13% accuracy improvement. Solves multilingual mixed document recognition challenges.

• PP-StructureV3 — Complex Document Parsing
  Intelligently converts complex PDFs and document images into Markdown and JSON files that preserve original structure. Outperforms numerous commercial solutions in public benchmarks. Perfectly maintains document layout and hierarchical structure.

• PP-ChatOCRv4 — Intelligent Information Extraction
  Natively integrates ERNIE 4.5 to precisely extract key information from massive documents, with 15% accuracy improvement over previous generation. Makes documents "understand" your questions and provide accurate answers.
  In addition to providing an outstanding model library, PaddleOCR 3.0 also offers user-friendly tools covering model training, inference, and service deployment, so developers can rapidly bring AI applications to production.

[hwf1324]

I have wrap https://github.com/RapidAI/RapidOCR into a test add-on: https://github.com/hwf1324/RapidOCR/releases/download/add-on-0.1.0/rapidOCR-0.1.0.nvda-addon

Please note that this plugin is for testing the Paddle OCR ONNX model only.

I apologize that I am unable to write more detailed documentation.