feat: add do_reading_order option to PdfPipelineOptions#3233
feat: add do_reading_order option to PdfPipelineOptions#3233Frank-Schruefer wants to merge 4 commits intodocling-project:mainfrom
Conversation
Add do_reading_order (default True) to PdfPipelineOptions, analogous to the existing do_table_structure and do_ocr flags. When set to False, ReadingOrderModel skips the ReadingOrderPredictor and keeps the element order produced by the layout postprocessor as-is. This is useful for PDFs where the reading-order predictor produces incorrect results — for example scanned pages that also carry a native text layer, where the many resulting small orphan clusters confuse the graph-based predictor. Signed-off-by: stone <frank.schruefer@t-online.de>
|
❌ DCO Check Failed Hi @Frank-Schruefer, your pull request has failed the Developer Certificate of Origin (DCO) check. This repository supports remediation commits, so you can fix this without rewriting history — but you must follow the required message format. 🛠 Quick Fix: Add a remediation commitRun this command: # Unable to auto-generate remediation message. Please check the DCO check details.
git push🔧 Advanced: Sign off each commit directlyFor the latest commit: git commit --amend --signoff
git push --force-with-leaseFor multiple commits: git rebase --signoff origin/main
git push --force-with-leaseMore info: DCO check report |
Merge ProtectionsYour pull request matches the following merge protections and will not be merged until they are valid. 🟢 Enforce conventional commitWonderful, this rule succeeded.Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
|
|
Related Documentation 2 document(s) may need updating based on files changed in this PR: Docling How does Docling reconstruct reading order without using a large language model (LLM) call in an unsupervised manner?View Suggested Changes@@ -32,6 +32,12 @@
## Why It Works Without LLMs
Spatial positioning encodes reading order for most documents (top-to-bottom, left-to-right conventions), and the ordering problem reduces to a **topological sort over a directed graph**—something DFS solves efficiently and deterministically.
+## Configuration
+
+Reading order prediction can be controlled via the `do_reading_order` flag in `PdfPipelineOptions` (defaults to `True`). Setting `do_reading_order=False` disables the `ReadingOrderPredictor` entirely, and elements retain the order produced by the layout postprocessor.
+
+Disabling reading order prediction is useful for certain document types—particularly scanned PDFs with native text layers—where the graph-based predictor may produce incorrect results due to many small orphan text clusters.
+
## Known Limitations
- The 15% dilation threshold is fixed and **not user-configurable**, which can cause issues with complex multi-column layouts.
- Reading order quality depends heavily on the accuracy of upstream layout detection.What are the detailed pipeline options and processing behaviors for PDF, DOCX, PPTX, and XLSX files in the Python SDK?View Suggested Changes@@ -4,6 +4,7 @@
- `from_formats`: Supported input formats include `docx`, `pptx`, `html`, `image`, `pdf`, `asciidoc`, `md` (including `txt`, `text`, `qmd`, `rmd`), `csv`, `xlsx`, `xml_uspto`, `xml_jats`, `xml_xbrl`, `mets_gbs`, `json_docling`, `audio`, `vtt`, `latex`
- `to_formats`: Supported output formats include `md`, `json`, `yaml`, `html`, `html_split_page`, `text`, `doctags`, `vtt`
- `pdf_backend`: Allowed values: `pypdfium2`, `docling_parse`, `dlparse_v1`, `dlparse_v2`, `dlparse_v4` (default: `docling_parse`)
+ - `do_reading_order` (default True): Enable reading-order prediction to reorder document elements into logical reading sequence; when disabled, elements retain the order produced by the layout postprocessor; disabling can improve results for scanned PDFs with native text layers and many small orphan clusters
- `do_ocr` (default True): Use OCR
- `force_ocr`: Replace existing text with OCR-generated text
- `ocr_engine`, `ocr_lang`: OCR engine and language optionsNote: You must be authenticated to accept/decline updates. |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
There was a problem hiding this comment.
@Frank-Schruefer thanks for this PR, I can see the point of the suggested change. I would like to ask you however to align a bit with conventions we have, by:
- removing the
skip_predictionin the ReadingOrderOptions again - instead, create an
enabledargument on theReadingOrderModel.__init__similar to this, with a default value of true. - have
do_reading_ordercontrol theenabledarg.
Thanks.
Replace skip_prediction option in ReadingOrderOptions with an enabled parameter in ReadingOrderModel.__init__, consistent with AutoOcrModel and other pipeline models. The do_reading_order flag in PdfPipelineOptions now controls this enabled argument directly. Signed-off-by: stone <frank.schruefer@t-online.de>
|
@Frank-Schruefer Please re-run a few times: |
Signed-off-by: stone <frank.schruefer@t-online.de>
There was a problem hiding this comment.
@Frank-Schruefer sorry for walking back on this. I recognize that this does not follow actual enabled semantics now. It still does the full caption, footnote and merge mapping. If that is your intention, then I agree this should have been more of a "skip reordering" option instead of an enabled flag for the full model.
Seeing this I would suggest to re-instate the previous option but:
- Make it a "positive" option flag of the reading order model options (such as
reorder_elements: bool = True). Then let thedo_reading_orderpipeline flag control that option field. Theskip_predictionfield you proposed originally was misleading me here.
Replace constructor-level `enabled` flag with `reorder_elements: bool = True` field in ReadingOrderOptions, consistent with the options-based pattern used throughout the codebase. Signed-off-by: Frank-Schruefer <frank.schruefer@sfs-software.de> Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Summary
This draft PR adds a
do_reading_orderflag toPdfPipelineOptions, analogous to the existingdo_table_structureanddo_ocroptions.Motivation
The
ReadingOrderPredictorworks well for most PDFs, but produces incorrect results for certain document types — in particular, scanned PDFs that also carry a native text layer. On such pages, each line of text typically becomes its own small orphan cluster, and the graph-based predictor struggles with 50–100 small clusters and reorders them incorrectly.With
do_reading_order=False, elements retain the order produced by the layout postprocessor. Combined with spatial cell sorting (sort_cells_spatialinBaseLayoutOptions), this gives correct reading order for the affected pages.Changes
PdfPipelineOptions.do_reading_order: bool = True— new flag, defaults to True (no change in existing behaviour)ReadingOrderOptions.skip_prediction: bool = False— internal flag on the modelStandardPdfPipelinewiresdo_reading_order→skip_predictionOpen question
Is this the right approach, or would you prefer a different mechanism to handle cases where the reading-order predictor produces poor results? Happy to adjust based on feedback.