Write a contract for every source
Record the expected file or interface, grain, keys, time fields, cadence, late-arriving behavior, corrections, owner, and approved control totals. The contract should also state what happens when the source is absent, duplicated, delayed, or structurally changed.
A pipeline is not ready because it loaded. It is ready when the organization can tell whether the loaded record is complete enough for its intended use.
Reconcile within domains first
Claims should reconcile to approved claim counts and financial amounts by status and period. Census should reconcile to location and day snapshots. Labor should reconcile to worked or paid hours under the selected definition. Finance should reconcile to the approved ledger or reporting extract.
Cross-domain ratios should not be evaluated until their component domains pass the controls required for that use.
- Row and distinct-key counts
- Duplicate and orphan checks
- Amount and hour totals
- Date-range completeness
- Late and corrected record counts
- Source-to-model variance
Make timing explicit
Operational date, service date, submission date, adjudication date, payment date, work date, posting date, and reporting period can produce different but legitimate answers. Each metric must state which clock it uses.
Restatements should be visible. A prior period that changed after late claims or finance close is not the same as a stable historical snapshot.
Route failures to owners
Failed controls should stop or qualify publication according to consequence. The record needs severity, affected period, source owner, analytical owner, disposition, and resolution evidence.
This turns data quality from an informal analyst task into an operating control that can be inspected and improved.
Primary sources
Need help with this system?
Reporting and BI Rescue turns the diagnostic into a bounded repair, rationalization, and operating baseline.
Review Reporting and BI Rescue