What this study cannot conclude
—
Bland-Altman bias and 95% limits of agreement (per field)
Each strip plots the per-ticker % difference (candidate − reference,
relative to the paired mean). The solid line is the mean bias; the dashed
lines are the 95% limits of agreement (bias ± 1.96·SD). A clean
study has the solid line near zero, narrow LoA, and no visible
heteroscedasticity.
Per-field agreement statistics
MAPE / p95 are sample-level dispersion. CCC is Lin's Concordance Correlation
Coefficient (1.0 = perfect agreement). Bias is the Bland-Altman mean; LoA
is its 95% interval. Wilcoxon p tests H0: median bias = 0
(significant at p < 0.01). Tolerance is a DCF-materiality threshold,
not "acceptable noise". Agreement tier rule: HIGH requires
CCC ≥ 0.99, |bias| ≤ 1× tolerance, ≥ 90 % of names within the
tolerance band, and no significant Wilcoxon bias (p ≥ 0.01);
MEDIUM relaxes to CCC ≥ 0.95 and |bias| ≤ 2× tolerance; otherwise
LOW. The ≥ 90 %-within-band condition is a gate for HIGH, not a
hidden criterion — a field can pass CCC/bias yet miss HIGH on it.
Denominators: the outlier / %-delta columns divide by
|reference|; the Bland-Altman strips divide by the paired mean —
different questions, not interchangeable. Pairs whose reference is
near zero relative to the field scale are split out (percentage
meaningless) and counted, not silently dropped. Where a field carries
a documented candidate-side artefact (e.g. multi-class share
summation), an ex→TIER chip shows the tier with those
names removed. The ex-known view is a scalpel for point
artefacts, not a cure for systemic bias: where a field disagrees by
a broad bias across the whole universe, removing a few named tickers
leaves the tier unchanged — shares outstanding stays LOW on the
full universe even after excluding every multi-class name, because
the disagreement is systemic, not a handful of exceptions.
| Field |
N |
Cov % |
MdAPE % |
IQR % |
MAPE %† |
p95 % |
CCC |
Bias % |
LoA % [low, high] |
Spearman ρ |
Wilcoxon p |
Tol % |
Tier |
Why each tier? Hover the chip — the explicit reason is in the
tier_reason attribute on every row.
Heatmap — signed % delta per ticker × field
Continuous colour scale — saturation grows with |delta|. Green ≤ tolerance,
amber tolerance–5%, red > 5%. Empty cell = the field is unavailable on
one of the two providers (recorded as missing rather than penalised).
Outliers (|delta| > 5 %)
Each entry is a ticker × field pair where the candidate disagrees with
the SEC EDGAR reference by more than 5 %. The most likely root causes
are (i) fiscal-window misalignment between yfinance's vendor TTM and
EDGAR's latest 10-Q sum, (ii) a restatement not yet ingested, or
(iii) different treatment of stock-based compensation in FCF.
| Ticker |
Field |
EDGAR (ref) |
yfinance (cand) |
Delta |
Tol % |