Arya Bank — data quality & reconciliation
Fictional bank, synthetic data: 1,000 customers, 1,366 accounts, 50,285 transactions for September 2026, with known defects injected on purpose. Checked with SQL rules in DuckDB and Great Expectations, then reconciled against the reporting pipeline. Updated 12 Oct 2026, 00:46 UTC.
Data quality score
98.86
of 100 · worst day 17 Sep (96.18)
Rules exact vs ground truth
12/12
no misses, no false positives
Reconciliation
7/7
source → report checks pass
Pipeline bugs caught
6/6
68/68 tests green on the correct pipeline
Completeness
96.0
Uniqueness
99.7
Validity
98.83
Integrity
99.79
Timeliness
99.96
Consistency
98.88
Failed transaction rows per day
Rows breaking a row-level rule (bad currency, non-positive amount, unknown account, duplicate copy). The 17 Sep spike is the injected feed incident. Rows dated after the cutoff are not shown.
Show as table
| Date | Rows | Failed | Score |
|---|---|---|---|
| 2026-09-01 | 1,721 | 7 | 99.59 |
| 2026-09-02 | 1,658 | 2 | 99.88 |
| 2026-09-03 | 1,657 | 6 | 99.64 |
| 2026-09-04 | 1,693 | 6 | 99.65 |
| 2026-09-05 | 1,670 | 8 | 99.52 |
| 2026-09-06 | 1,729 | 8 | 99.54 |
| 2026-09-07 | 1,655 | 7 | 99.58 |
| 2026-09-08 | 1,659 | 4 | 99.76 |
| 2026-09-09 | 1,648 | 3 | 99.82 |
| 2026-09-10 | 1,707 | 12 | 99.3 |
| 2026-09-11 | 1,630 | 4 | 99.75 |
| 2026-09-12 | 1,694 | 6 | 99.65 |
| 2026-09-13 | 1,704 | 8 | 99.53 |
| 2026-09-14 | 1,716 | 6 | 99.65 |
| 2026-09-15 | 1,668 | 7 | 99.58 |
| 2026-09-16 | 1,645 | 4 | 99.76 |
| 2026-09-17 | 1,726 | 66 | 96.18 |
| 2026-09-18 | 1,694 | 11 | 99.35 |
| 2026-09-19 | 1,610 | 7 | 99.57 |
| 2026-09-20 | 1,680 | 7 | 99.58 |
| 2026-09-21 | 1,707 | 10 | 99.41 |
| 2026-09-22 | 1,592 | 6 | 99.62 |
| 2026-09-23 | 1,695 | 5 | 99.71 |
| 2026-09-24 | 1,683 | 10 | 99.41 |
| 2026-09-25 | 1,720 | 8 | 99.53 |
| 2026-09-26 | 1,616 | 8 | 99.5 |
| 2026-09-27 | 1,666 | 9 | 99.46 |
| 2026-09-28 | 1,673 | 7 | 99.58 |
| 2026-09-29 | 1,675 | 7 | 99.58 |
| 2026-09-30 | 1,674 | 6 | 99.64 |
Quality rules (SQL)
Each rule is graded against the generator's manifest of injected defects.
| Rule | Dimension | Check | Failed | Pass rate | vs ground truth |
|---|---|---|---|---|---|
UNQ-01 | Uniqueness | Each txn_id appears once transactions | 150 / 50,285 | 99.7% | ✓ exact 150 found / 150 injected |
CMP-01 | Completeness | Customer has email, phone and KYC status customers | 40 / 1,000 | 96.0% | ✓ exact 40 found / 40 injected |
VAL-01 | Validity | Currency is INR or a currency with an FX rate transactions | 60 / 50,285 | 99.88% | ✓ exact 60 found / 60 injected |
VAL-02 | Validity | Amount is a number greater than zero transactions | 30 / 50,285 | 99.94% | ✓ exact 30 found / 30 injected |
VAL-03 | Validity | Email is well formed customers | 25 / 1,000 | 97.5% | ✓ exact 25 found / 25 injected |
VAL-04 | Validity | Phone is a 10-digit Indian mobile number (starts 6-9) customers | 20 / 1,000 | 98.0% | ✓ exact 20 found / 20 injected |
INT-01 | Integrity | Transaction belongs to a known account transactions | 25 / 50,285 | 99.95% | ✓ exact 25 found / 25 injected |
INT-02 | Integrity | Account's branch exists in the branch master accounts | 5 / 1,366 | 99.63% | ✓ exact 5 found / 5 injected |
TML-01 | Timeliness | Transaction is not dated after the extract cutoff transactions | 20 / 50,285 | 99.96% | ✓ exact 20 found / 20 injected |
CON-01 | Consistency | Savings account closing balance is not negative accounts | 10 / 1,366 | 99.27% | ✓ exact 10 found / 10 injected |
CON-02 | Consistency | Closing balance = opening + credits - debits (successful, valid transactions) accounts | 25 / 1,366 | 98.17% | ✓ exact 25 found / 25 injected |
CON-03 | Consistency | Customers under 18 cannot hold a CURRENT account customers | 8 / 1,000 | 99.2% | ✓ exact 8 found / 8 injected |
Reconciliation: source → report
Expected figures are recomputed from the raw extract with independent SQL, then compared with the pipeline output.
| Check | What it proves | Result | Expected / actual |
|---|---|---|---|
REC-01 | Every raw row is either loaded or quarantined | ✓ pass | 50285 raw rows 50000 loaded + 285 quarantined = 50285 |
REC-02 | Each valid transaction is loaded exactly once | ✓ pass | 50000 valid transactions 0 missing, 0 unexpected, 0 duplicated |
REC-03 | Quarantine counts per reason match the quality rules | ✓ pass | DUPLICATE 150, INVALID_CURRENCY 60, NON_POSITIVE_AMOUNT 30, UNKNOWN_ACCOUNT 25, FUTURE_DATED 20 DUPLICATE 150, FUTURE_DATED 20, INVALID_CURRENCY 60, NON_POSITIVE_AMOUNT 30, UNKNOWN_ACCOUNT 25 |
REC-04 | Daily control totals (count, credits, debits) match the source | ✓ pass | every day matches 0 of 30 report days differ |
REC-05 | Every branch-day row matches the source | ✓ pass | every branch-day matches 0 branch-day rows differ (report has 630) |
REC-06 | Computed closing balance per account matches the source ledger | ✓ pass | all accounts match 0 accounts differ |
REC-07 | Balance breaks in the report are exactly the known source breaks | ✓ pass | 25 accounts (rule CON-02) 25 accounts flagged by report |
Planted pipeline bugs
The whole test suite is rerun with one bug switched on at a time. A bug counts as caught when at least one test fails.
| Bug | Defect | Result | Caught by |
|---|---|---|---|
PIPE-01 | Duplicates not removed A re-sent transaction is counted twice in the daily totals. | ✓ caught | Unit, Reconciliation 7 failing tests |
PIPE-02 | FX uses the latest rate for every day Foreign remittances are converted at the 30 Sep rate instead of the rate on their own date. | ✓ caught | Unit, Reconciliation 9 failing tests |
PIPE-03 | Report date taken in UTC, not IST Payments made between 00:00 and 05:30 IST land on the previous day's report. | ✓ caught | Unit, Reconciliation 9 failing tests |
PIPE-04 | Inner join to branch master Transactions of accounts with an unknown branch vanish without being quarantined. | ✓ caught | Unit, Reconciliation 7 failing tests |
PIPE-05 | Failed and reversed payments counted Totals include money that never moved. | ✓ caught | Unit, Reconciliation 9 failing tests |
PIPE-06 | Paise dropped Amounts are floored to whole rupees before totalling. | ✓ caught | Unit, Reconciliation 9 failing tests |
Great Expectations
| Table | Column | Expectation | Result | Unexpected | Sample |
|---|---|---|---|---|---|
| transactions | txn_id | not be null | ✓ pass | 0 | |
| transactions | txn_id | be unique | ✗ fail | 300 | T00042777, T00032241, T00043108 |
| transactions | account_id | match regex | ✓ pass | 0 | |
| transactions | currency | be in set | ✗ fail | 48 | usd, usd, RS |
| transactions | currency | not be null | ✗ fail | 12 | nan, nan, nan |
| transactions | amount_num | be between | ✗ fail | 30 | 0.0, -4991.59, 0.0 |
| transactions | direction | be in set | ✓ pass | 0 | |
| transactions | status | be in set | ✓ pass | 0 | |
| transactions | channel | be in set | ✓ pass | 0 | |
| transactions | txn_ts_utc | be between | ✗ fail | 20 | 2026-10-10 13:30:00, 2026-10-06 07:30:00, 2026-10-10 00:30:00 |
| customers | customer_id | be unique | ✓ pass | 0 | |
| customers | email | not be null | ✗ fail | 14 | nan, nan, nan |
| customers | email | match regex | ✗ fail | 25 | zarna x@example.com, unni@, xavier x@example.com |
| customers | phone | not be null | ✗ fail | 13 | nan, nan, nan |
| customers | phone | match regex | ✗ fail | 20 | 5987654321, 5987654321, 5987654321 |
| customers | kyc_status | not be null | ✗ fail | 13 | nan, nan, nan |
| customers | kyc_status | be in set | ✓ pass | 0 | |
| customers | date_of_birth | match regex | ✓ pass | 0 | |
| accounts | account_id | be unique | ✓ pass | 0 | |
| accounts | account_type | be in set | ✓ pass | 0 | |
| accounts | branch_code | be in set | ✗ fail | 5 | ARYB9999, ARYB9999, ARYB9999 |
| accounts | currency | be in set | ✓ pass | 0 |
Quarantine
Rows the pipeline refused to load, with the reason. Nothing is dropped silently.
| Reason | Rows |
|---|---|
DUPLICATE | 150 |
FUTURE_DATED | 20 |
INVALID_CURRENCY | 60 |
NON_POSITIVE_AMOUNT | 30 |
UNKNOWN_ACCOUNT | 25 |