Find duplicate rows caused by overlapping PDF extraction page ranges

Find the overlapping page before comparing business values

Compare source coverage before deleting anything so you remove extraction duplicates rather than legitimate repeated records.

For multipage invoices, use ParseMill to review extracted invoice data beside the source and export separate summary and line-item sheets. Include a page break in your trial and check that every required line made it through. Other report types need their own fit assessment.

A file name such as “part two” is too weak to establish its starting page. Recover the actual configured range or inspect the source references. If a batch has no dependable coverage record, isolate it for review. Do not compensate by deleting every matching amount or description from the combined output.

Make a page-range manifest

Microsoft: Pdf.Tables supports explicit page limits. Microsoft: Table.Distinct uses specified fields to identify duplicates; the fields chosen therefore matter to what disappears.

1. List each batch, the exact source-file version, its first and last physical pages, and its output file. Record whether the range endpoints are inclusive.
2. Expand the ranges into a coverage list. Page 5 should show two contributing batches in the example; pages 1–4 and 6–10 should show one.
3. Compare the page-5 rows by source row identity, not just their extracted cell values. Resolve disagreements between the two outputs before selecting one copy.
4. Retain one reviewed contribution from each intended source row. Mark the unused batch contribution as duplicate processing coverage and keep the original batch outputs available.

Reconcile the boundary explicitly

Synthetic example: page 5 contains three detail rows. The two runs return the following page contributions.

Appending both batches yields 33 rows. Removing the three confirmed second contributions gives 30 retained rows. Two separate rows elsewhere both say “Delivery, 0.00”; retain both if they occupy different source locations. Their identical values do not make either an extraction overlap. If page 5 produces different row counts in A and B, the expected total of 30 is only a fixture expectation, not a reason to force the real result to match.

Resolve mismatches without hiding them

Suppose batch A joins a wrapped description while batch B splits it. Choose neither version merely because it has more rows. Compare the visible source, identify the intended record boundary and record a manual reconstruction or unsupported-layout outcome. A second run can introduce a new interpretation rather than a redundant copy.

Likewise, a repeated page inside the PDF is a different condition. The page-range manifest would show one visit to each physical position even if two positions contain the same source page. Investigate that document assembly issue separately so the correction retains a truthful explanation.

Close the coverage check

The final record should explain every intended page, every overlap, each excluded contribution and any unresolved boundary row. Reconcile input contributions against retained and excluded rows, then inspect the first and last retained record from each batch. Confirm that each source link still points into the original complete PDF, with any batch-relative page number translated correctly.

For the next run, use nonoverlapping ranges when possible and preserve the manifest with the export. Use the multipage table checklist to review continuity as well as coverage. ParseMill does not turn a page-count match into evidence of complete or accurate extraction.