Keep reviewer disagreements visible in an extraction study

Disagreement is useful evidence

Keep the differences visible so the team can resolve the interpretation before using the test to choose a tool.

In ParseMill, select an extracted invoice field to inspect its source, correct the value and export the reviewed workbook. Try your own invoices as well as the interactive sample. Compare upload, checking and correction time with your current way of building the same spreadsheet.

Keep the exact source version, cell location and field definition with the dispute. Without those, reviewers may be comparing different pages, revisions or interpretations of the requested field rather than disagreeing about the same observation.

Copy a disagreement record

The entries are invented. The first remains unresolved because neither reading has sufficient evidence. The second can be corrected from a clear heading. The third shows that a more specific interpretation is not automatically better when the source does not support it.

W3C: Provenance model primer models responsible agents and derivations. A review log can retain who proposed each reading and the evidence behind the accepted result.

Adjudicate without erasing the alternatives

1. Record each reading independently, including the reason and source location. Keep the tested output separate if it may influence the discussion.
2. Verify that everyone is viewing the same source revision and applying the same field definition. Resolve those mismatches first.
3. Inspect additional source context or obtain a clearer document. Use an appropriately qualified adjudicator when the dispute concerns technical interpretation.
4. Record accepted, corrected or unresolved status with the rationale, reviewer and date. Recheck any downstream fields affected by the decision.

W3C: Data on the Web Best Practices recommends version indicators and history. Bind the adjudication to the exact source and output versions so later changes do not inherit an obsolete decision.

Avoid false consensus

A majority vote can settle a workflow decision, but it does not make an unreadable glyph legible. If evidence cannot distinguish two readings, keep the field unresolved and state the consequence for use. Do not average competing numeric readings as a transcription repair. An average would be a new value supported by neither reading.

When a disagreement reveals an unclear field definition, update the definition and revisit earlier records affected by it. Otherwise, the final register may apply different interpretations depending on which reviewer handled each row.

Verify the resolution trail

The fixture ends with S8 on hold, S9 assigned cm from the heading, and S10 recorded as blank in source. The log preserves the rejected alternatives and why they were rejected. Another reviewer should be able to follow that reasoning without relying on memory.

Use a disagreement log during a ParseMill evaluation. It helps separate tool defects, ambiguous source documents and review-policy questions, making the resulting improvement work more specific and useful.