Errata in the source research¶
Why this page exists¶
This project began from a literature survey produced with AI assistance and reviewed by a human. Both stages let errors through, which is normal and is the reason a correction record exists rather than an apology.
The survey lives in docs/research/ and is never edited. It is a record of
what was believed at the start, and rewriting it would destroy the evidence that
these corrections were needed. Everything wrong with it is listed here instead.
Each entry states what the research says, what is actually true, how that was verified, and what it changes.
1. RAPM was placed in the wrong tier¶
The research says, in CLAUDE.md's summary tables, that RAPM is Tier 1 at
85-95%.
Actually true: RAPM scores 80-85%. The methodology defines Tier 1 as 85-95%
and Tier 2 as 75-85%, so 80-85% is Tier 2. Both
nba_metrics_reliability_report.md and the ranking table in
nba_metrics_detailed_report.md file it under Tier 2 correctly. Only the summary
promoted it.
Verified by reading which tier heading each report files RAPM under, and
comparing against the boundaries in nba_metrics_reliability_methodology.md.
What it changes: RAPM is this project's ground truth. Overstating its reliability by a tier would have propagated into every downstream claim. Corrected in the project instructions.
2. Tier labels are applied inconsistently at the same score¶
The research says D-EPM is Tier 1 and D-RAPM is Tier 2.
Actually true: both are given the identical reliability of 80-85%. That band is Tier 2 by the methodology's own definition, so D-EPM's Tier 1 label is wrong.
| Metric | Reliability given | Tier given | Tier the boundaries require |
|---|---|---|---|
| O-EPM | 85-90% | Tier 1 | Tier 1 |
| D-EPM | 80-85% | Tier 1 | Tier 2 |
| O-RAPM | 80-85% | Tier 2 | Tier 2 |
| D-RAPM | 80-85% | Tier 2 | Tier 2 |
What it changes: it shows the tiers were assigned by impression and then given numbers, rather than derived from numbers. That is the strongest single argument for measuring reliability rather than citing it.
3. The 0.44 constant is mischaracterised¶
The research says PER is unreliable partly because "constants like 0.44 (for possession estimation) are outdated and may not reflect modern NBA realities".
Actually true: 0.44 is an empirical estimate of the fraction of free throw attempts that consume a possession. Not every free throw ends a possession. On a two-shot foul only the second does. On an and-one the field goal already ended it. On a three-shot foul only the third does. Technical free throws end nothing. The measured fraction lands near 0.44.
Verified against two archived sources in this project's own bibliography:
"The 0.44 estimates how many possessions free throws actually cost (most trips are two shots, but and-ones and technicals skew it)." (SRC-6)
"The 0.44 coefficient in the TS% formula estimates the portion of free throw attempts that represent actual possessions" (SRC-55)
Three things follow.
It is not arbitrary. It estimates a specific, measurable quantity.
It is not specific to PER. The same constant, for the same purpose, appears
in Pace, turnover rate, true shooting and usage rate, which the research places
in Tier 1 and Tier 2. PER's version is not a different or worse constant. The
Wikipedia formula for uPER, archived here as SRC-16, contains
0.44 × lgFTA / lgPF, identical in purpose.
It is not the two-versus-three adjustment. That logic, where three
two-pointers equal two three-pointers, lives in effective field goal
percentage, whose 0.5 × 3PM term counts a three as 1.5 field goals because
3 ÷ 2 = 1.5. The separate 2 × in the true shooting denominator is a scaling
that puts the result on the same footing as field goal percentage. Neither is
0.44.
What survives of the criticism of PER. Removing this leaves the real objections, which are enough on their own:
- Defence is captured only through steals and blocks, which the author concedes.
- No adjustment for teammates, opponents or role.
- The event weights themselves are fitted values, not derivable quantities.
- The league is renormalised to exactly 15 every season, forcing a fixed distribution regardless of how good the league actually was.
Measured across every available season¶
All 25 seasons hoopR publishes, 2002 through 2026, have now been measured.
| Value | |
|---|---|
| Lowest season (2023) | 0.4096 |
| Highest season (2004) | 0.4252 |
| Mean across 25 seasons | 0.4178 |
| Spread | 0.0156 |
| Mean gap from 0.44 | -0.0222 |
Two things follow, and the second is the more useful one.
Every season sits below 0.44, without exception. Not once in 25 years does the measured value approach the conventional one. The nearest is 2004 at 0.4252, still 0.0148 short.
The constant's error is larger than its drift. The spread across a quarter century is 0.0156, while the average distance from 0.44 is 0.0222. A fixed value of 0.4178 would therefore be closer to every season than 0.44 is to any of them. There is a slow downward trend, from roughly 0.42 in the 2000s to roughly 0.41 in the 2020s, which is worth a per-season value rather than a new constant, but even the crudest replacement beats the convention.
Coverage for 2002 is partial: 23,747 attempts against 50,000 to 69,000 in every later season. Its figure is reported for completeness and carries less weight.
The exclusion rule, and why it decides the answer¶
This measurement depends entirely on one choice, so it is stated plainly rather than buried. And-one, technical, flagrant and clear-path free throws are counted in the denominator as attempts, and never in the numerator as possession-enders, because none of them ends a possession. Together they are 14.9% of all attempts in 2023-24.
Dropping them from the denominator as well moves the 2023-24 figure from 0.4104 to 0.4823. The conventional 0.44 sits between those two numbers, which is worth noticing before assuming the convention is simply wrong.
The implemented choice is the one the formula needs.
Possessions = FGA + 0.44 x FTA + TOV consumes total free throw attempts,
including and-ones and technicals, so the coefficient must be the fraction of
all attempts that end a possession. The 0.4823 variant answers a different
question, namely what fraction of free throws that could end a possession
actually do, and substituting it into that formula would double-count.
Reproduce any of this with:
The measured value is now the project default¶
possession_coefficient.coefficient_for_season(season) returns the measured
figure for that season, or the pooled 0.4178 for a season with no measurement of
its own. Anything in this project that would otherwise reach for 0.44 uses that
instead.
CONVENTIONAL_COEFFICIENT is kept at 0.44, as the number to compare against
rather than the number to use. Removing it would make the comparison impossible
to state.
Two tests defend the finding rather than only recording it. One asserts that no measured season sits at or above 0.44, so a future measurement that contradicts the headline claim fails the build rather than quietly joining the table. The other reloads all 25 seasons, recomputes every coefficient, and compares against the published table, because a hardcoded finding that has silently drifted from its source is worse than no finding: it carries the authority of a measurement while no longer being one.
This is a finding specific to this project, not an established result. It depends on the exclusion rule stated above, on hoopR's event labelling, and on ESPN as the upstream source. Anyone reproducing it with different play-by-play should expect a different number, and the honest comparison is against their own measurement rather than against 0.44.
What it changes for this project. There is a legitimate version of the complaint, and it applies to every metric rather than to PER. The true fraction drifts with rule changes, and hard-coding 0.44 across a 25-season span bakes in an assumption nobody checks. This project has the play-by-play data to measure the coefficient per season instead. That is now the intended approach, and the fixed 0.44 becomes the baseline it is compared against.
The measured coefficient¶
Measured for the 2023-24 season: 0.410, against the conventional 0.44. That season's free throws ended a possession about 0.030 less often than the fixed constant assumes, roughly 7% lower in relative terms.
Of 57,076 free throw attempts, 57,071 could be classified from the columns hoopR provides, giving 23,421 possession-ending trips. And-one, technical, flagrant and clear-path free throws are excluded from that count by rule, not by matching specific foul-name strings: an and-one's field goal already ended the possession, a technical changes nothing, and a flagrant or clear-path foul lets the fouled team keep the ball regardless of whether the last shot goes in. Ruling them out this way, rather than pattern-matching foul names, is what lets the same logic hold across rule eras without being rewritten for each one, including the "transition take foul" and "away from play foul" rules the NBA added for 2023-24 itself.
The remaining 5 attempts (0.009% of the season) are a missed final free throw followed immediately by another shot, with no rebound row logged between them. The shooting team plainly kept the ball, but no label says so, and inferring it from the shooter's team would be a guess rather than a reading. Those attempts are excluded from both the numerator and the denominator.
An earlier version of this estimate excluded 29 attempts rather than 5. Its
lookahead stepped over substitutions only, and latched shut the moment any
other event appeared, so a foul call or a replay review sitting one event
before a perfectly clear rebound label made the whole trip unresolvable. The
excluded set was not random: six were coach's challenges and seven were runs
of three substitutions. Code review caught it. The lookahead now steps over
anything that is not a shot attempt, using hoopR's own shooting_play flag,
and stops at a shot because a rebound after a new shot belongs to that shot.
Correcting it moved the coefficient from 0.4102 to 0.4104, so the bias was
real but small. It is recorded here because a number whose error bar comes
from an undocumented bug is not a measurement.
Verified by running estimate_season_coefficient (see
src/pippen/data/possession_coefficient.py) against hoopR's real 2023-24
play-by-play file (play_by_play_2024.parquet, 614,447 rows) and checking
every intermediate count by hand: 32,268 free throw trips, of which 6,622 are
one-shot and-one or take-foul trips, 1,544 are technical, 169 are flagrant or
clear-path, 19,393 end on a made last shot, 4,005 end on a missed last shot
with a defensive rebound, and 506 continue on a missed last shot with an
offensive rebound.
What it changes: one season is a single measurement, not evidence of
drift across rule eras on its own. That comparison needs several seasons run
through estimate_coefficients_by_season, which this module also provides,
once more seasons are ingested. What the 2023-24 season already shows is that
0.44 is now a measured quantity in this project rather than an assumed one,
and the first measurement does not match it.
4. Citation integrity¶
Found while archiving all 157 citations. Full detail in
data/sources/MANIFEST.md.
| Problem | Detail |
|---|---|
| One paper, three identifiers | SRC-34, SRC-35 and SRC-89 are all arXiv 2601.15000, which inflates the apparent evidence base |
| One dead page, two identifiers | SRC-39 and SRC-42 are the same URL, and it returns 404 |
| EPM's headline figure has no live source | The 85-90% reliability rests on SRC-39, which is gone. Only a screenshot survives |
| PIPM methodology source is gone | SRC-112 returns 404 |
| Eight citations archive nothing | Every FiveThirtyEight URL now redirects to abcnews.com/politics. Seven supported CARMELO, one supported RAPTOR |
| One source is not a source | SRC-64 is a podcast episode description that appears to have been read as an article |
| One typo, not a dead link | SRC-38's URL carries one extra word in its slug. The corrected address is live |
5. What the research still gets right¶
This page is a correction record, not a dismissal. The survey's central conclusions hold up:
- Metrics built on lineup data with regularisation outperform box-score-only metrics by a wide margin.
- Team-level metrics are more reliable than individual ones, because aggregation cancels noise.
- Defence is the hardest thing to measure, and box-score defensive metrics are close to worthless.
- Multi-season pooling substantially reduces standard error.
- Low-minute players are noisy in every metric and need shrinkage.
Those are the load-bearing claims, and this project is built on them.