Reconcile schema changes before appending monthly CSV exports

A familiar file can have a new schema

Reconcile the column definitions first so a quick consolidation does not create a register that is harder to trust.

ParseMill exports reviewed invoices to Excel with separate summaries, line items, sources and review status. You can import an XLSX or CSV template and save a client setup. Run a real invoice through to the final workbook and check it in the application that receives your work.

Retain each monthly file unchanged and record its source month or delivery identity on every appended row. This makes a later mapping correction possible without guessing which records came from which input.

Use a schema comparison worksheet

The synthetic worksheet does not assume that absent January qualifiers mean unqualified results. Record an appropriate absent-field state based on what January actually supplied. Keep an unexpected field for review rather than discarding it because it was not in last month’s template.

Microsoft: Append queries documents name-based appending and nulls for missing columns. That behavior prevents positional alignment errors but still requires a reviewed semantic mapping.

Review before appending

1. Inventory names, order, types and representative values for each input. Record new, missing and renamed columns.
2. Create an explicit mapping to the target schema, including units and raw-versus-derived roles. Resolve ambiguous names before combining.
3. Append the mapped rows while retaining input-file identity and original record keys. Keep unmatched or unsupported fields in an exception table.
4. Reconcile each file’s contributed rows and compare boundary records with the originals. Check type conversions and missing-state handling in the combined output.

Microsoft: Table.TransformColumnTypes supports explicit column types. Apply types after the schema mapping is understood, and inspect errors rather than replacing them with silent blanks.

Watch changes that names do not reveal

A column called Result may switch from a fraction to a percentage number without being renamed. An identifier may become longer and exceed numeric precision. Review current source documentation and representative values instead of assuming a matching header guarantees compatibility.

A new column can also indicate a changed row model, such as one row per replicate instead of one per sample. Revisit record identity before deduplicating or summarizing the combined register.

Accept a reconciled combined dataset

The fixture passes when February’s reported result is mapped deliberately, its qualifier is retained, and January’s absence of that field is explicit. Each monthly contribution remains traceable and its row count is explained. Save the schema mapping version with the output. Include a list of fields that were renamed, added or left unresolved so the next monthly append can be reviewed efficiently.

When appending ParseMill exports over time, review schema changes before importing them into a cumulative register. A small mapping check protects the meaning of the entire combined dataset.