Use independent expected output for an extraction test

Build a reference that can disagree

Prepare an independent comparison so the trial tells you whether the tool found the right data.

In ParseMill, select an extracted invoice field to inspect its source, correct the value and export the reviewed workbook. Try your own invoices as well as the interactive sample. Compare upload, checking and correction time with your current way of building the same spreadsheet.

Freeze the source version before transcription. A reference built from a revised PDF cannot fairly evaluate output from the earlier version. Keep the expected-output revision separate from the extraction run identifier.

Use a two-pass reference worksheet

These synthetic entries show that independent review can find both disagreement and shared uncertainty. The final row should not acquire a guessed identifier merely to make the reference complete. Label unresolved reference fields so they are not treated as unquestionable ground truth.

Google: Dividing datasets explains why evaluation examples should be separate from development. The same independence principle motivates constructing expected values without letting the tested output dictate them.

Make the reference reproducible

1. Write the field definitions and comparison rules first. Specify how qualifiers, whitespace, units and missing states will be represented.
2. Transcribe directly from the frozen source, with row identity and evidence locations. Keep the parser output out of view during this first pass.
3. Perform a second source-based pass or independent review. Record disagreements, adjudication evidence and remaining ambiguities.
4. Version the finalized reference and use it unchanged for the declared comparison. If an error is later found in the reference, record the correction and rerun affected results.

W3C: Data on the Web Best Practices recommends version indicators and history for data. A reference dataset needs the same clarity so a score can be tied to the exact expected output used.

Do not conceal reference uncertainty

A blurry cell may be impossible to resolve from the available source. Exclude it from an exact-value score only under a stated rule and report the exclusion count. Keep it in coverage and exception reporting so the difficult case does not disappear from the evaluation or become an unreported exclusion from the denominator.

When reviewers disagree, distinguish a transcription mistake from a genuinely ambiguous source. The remedy may be a clearer document rather than a vote between two plausible readings.

Verify the evaluator as well as the extraction

The fixture’s reference is ready only after the unit conflict and footnote omission have been resolved or explicitly marked unresolved. Save the source identity, reference version and comparison rules with any reported results.

Use this method when evaluating ParseMill on your own documents. An independently prepared expected register makes it easier to identify real improvements, remaining limitations and the review effort required for your workflow.