What you can do today

  • In Optimize, create a processing policy with immutable schema versions and expected field results.
  • Keep development, holdout and shadow cases separate, with distinct source documents and declared layout families.
  • Measure correct, incorrect and missing fields, evidence availability and OCR warnings.
  • Optimize supports promotion after held-out and shadow checks, explicit application and rollback to a qualified rule version.
  • In Correct & learn, save immutable corrections and propose literal-label aliases from learning feedback; corrected values and graph facts are not hardcoded into extraction results.
  • Freeze a feedback candidate before evaluating at least two unused held-out documents from distinct reviewer-declared families. Each held-out document qualifies only one evaluated candidate within that learning program.
  • Require measured improvement without regressions before owner acceptance. Apply accepted feedback rules explicitly or suspend them; every resulting document still requires review.
  • Retain evaluation and decision history. Changed sources or approved organization context invalidate feedback qualification.

Test a new supplier label before reusing it

Add a label alias in a new rule version. Compare the candidate against annotated holdout and shadow documents, inspect failures, then promote the tested version. Reuse the previous qualified version if a later change performs worse.

Supported scope and limits

  • Execution currently uses literal-label extraction rules. It does not generate or run arbitrary scripts, call model providers or schedule business processes.
  • Stored confirmation and correction rewards support evaluation; no model weights are trained and online reinforcement learning is not implemented.
  • Shadow mode in Optimize compares extraction rules without external writes; it does not simulate an entire accounting workflow.
  • Not every task can become a deterministic script. Passing a small annotated set does not establish general accuracy.
  • Layout families and expected values are supplied by the reviewer. Separate examples and accurate annotations matter.
  • Feedback learning supports up to 100 approved organization knowledge records in its context. It does not run embeddings or GraphRAG.

Planned extensions

  • Qualified model comparisons with actual provider receipts and bounded spend.
  • Evaluated correction suggestions beyond literal-label aliases.
  • Broader registered operations and workflow-level outcome comparisons.
  • Production drift sampling and controlled rollout across document cohorts.

How to check the result

  • Keep held-out and shadow documents outside rule development.
  • Inspect wrong values, missing values and source coverage separately.
  • Measure review effort alongside processing results; lower token use alone is not a business outcome.