`pippen.data.hoopr`¶

Bulk historical download from the hoopR NBA data repository, which publishes one Parquet file per season under CC BY 4.0.

Downloads are atomic. Each file streams to a temporary sibling, has its Parquet footer validated, and only then replaces the target. A crash partway through cannot leave a truncated file where a later run would skip it as already present, which would silently poison every calculation downstream.

pippen.data.hoopr ¶

Downloader for hoopR's bulk historical NBA data.

hoopR (part of the SportsDataverse project) publishes ESPN-sourced NBA data as one Parquet file per dataset per season, hosted in the hoopR-nba-data GitHub repository::

https://github.com/sportsdataverse/hoopR-nba-data

Attribution and licence That repository is licensed CC BY 4.0, which permits redistribution with attribution. :data:ATTRIBUTION holds the required credit line; anything this project publishes that is derived from hoopR data must carry it (see docs/research/06_data_sources.md).

Season numbering season throughout this module is the season's end year: 2024 means the 2023-24 season. This matches pippen.paths.season_file.

Known limitation -- load-bearing for the rest of the pipeline hoopR's play_by_play dataset is ESPN-sourced and records only event participants (athlete_id_1/2/3 columns). It has no on-court lineup column. RAPM needs the full five-man lineup for every possession, so it cannot be built from this dataset alone; that requires pbpstats running against NBA API data instead. This module only downloads the file -- it makes no claim about what can be computed from its contents.

File layout Most datasets nest their Parquet files under a parquet/ subdirectory (nba/{dataset}/parquet/{dataset}_{season}.parquet); at least one does not. Rather than hard-coding one guess, every download probes both layouts with a HEAD request and uses whichever one answers.

One dataset's *directory* name diverges from its file-name prefix:
``play_by_play`` is published under ``nba/pbp/parquet/``, not
``nba/play_by_play/parquet/``, even though the files inside it are still
named ``play_by_play_{season}.parquet``. Confirmed live against the
repository on 2026-09-10 -- the URL pattern in the project's own data
source notes does not account for this, and probing alone cannot recover
it, since ``nba/play_by_play/`` simply does not exist. See
:data:`_REMOTE_DIRECTORY`.

TransientDownloadError ¶

Bases: Exception

A network failure worth retrying: timeout, connection reset, or 5xx.

Anything else -- a 404, a malformed URL, a disk error while writing -- is treated as permanent for the attempt and is not retried.

Source code in src/pippen/data/hoopr.py

class TransientDownloadError(Exception):
    """A network failure worth retrying: timeout, connection reset, or 5xx.

    Anything else -- a 404, a malformed URL, a disk error while writing -- is
    treated as permanent for the attempt and is not retried.
    """

DownloadResult `dataclass` ¶

Outcome of trying to get one season of one dataset onto disk.

Attributes:

Name	Type	Description
`dataset`	`str`	The hoopR dataset name.
`season`	`int`	Season end year, e.g. 2024 for the 2023-24 season.
`status`	`DownloadStatus`	`"downloaded"` if fetched this run, `"skipped"` if a valid file already existed, or `"failed"` (see `reason`).
`path`	`Path \| None`	Where the file ended up on disk. Set for `"downloaded"` and `"skipped"`, `None` for `"failed"`.
`reason`	`str \| None`	Human-readable cause of failure. Set only when `status` is `"failed"`.

Source code in src/pippen/data/hoopr.py

@dataclass(frozen=True)
class DownloadResult:
    """Outcome of trying to get one season of one dataset onto disk.

    Attributes:
        dataset: The hoopR dataset name.
        season: Season end year, e.g. 2024 for the 2023-24 season.
        status: ``"downloaded"`` if fetched this run, ``"skipped"`` if a
            valid file already existed, or ``"failed"`` (see ``reason``).
        path: Where the file ended up on disk. Set for ``"downloaded"`` and
            ``"skipped"``, ``None`` for ``"failed"``.
        reason: Human-readable cause of failure. Set only when ``status`` is
            ``"failed"``.
    """

    dataset: str
    season: int
    status: DownloadStatus
    path: Path | None = None
    reason: str | None = None

download_season ¶

download_season(dataset, season, *, force=False, timeout=_DEFAULT_TIMEOUT_SECONDS, session=None)

Download one season of one hoopR dataset into the raw data stage.

The file is streamed to a temporary path beside the destination and only moved into place after it passes an integrity check, so a crash or a truncated transfer never leaves a half-written file at the final path.

Parameters:

Name	Type	Description	Default
`dataset`	`str`	hoopR dataset name, e.g. `"play_by_play"`. Must be a member of :data:`DATASETS`.	required
`season`	`int`	Season end year, e.g. 2024 for the 2023-24 season.	required
`force`	`bool`	When true, re-download even if a valid file already exists at the destination. When false (the default), an existing file that passes the integrity check is left alone.	`False`
`timeout`	`float`	Per-request timeout in seconds, applied to every HTTP call this makes.	`_DEFAULT_TIMEOUT_SECONDS`
`session`	`Session \| None`	HTTP session to issue requests on. A short-lived session is created and closed automatically when omitted; callers doing many downloads should pass one in to reuse connections (see :func:`download_seasons`).	`None`

Returns:

Name	Type	Description
`A`	`DownloadResult`	class:`DownloadResult` describing what happened to this one file.

Raises:

Type	Description
`ValueError`	If `dataset` is not a recognised hoopR dataset name.

Source code in src/pippen/data/hoopr.py

def download_season(
    dataset: str,
    season: int,
    *,
    force: bool = False,
    timeout: float = _DEFAULT_TIMEOUT_SECONDS,
    session: requests.Session | None = None,
) -> DownloadResult:
    """Download one season of one hoopR dataset into the ``raw`` data stage.

    The file is streamed to a temporary path beside the destination and only
    moved into place after it passes an integrity check, so a crash or a
    truncated transfer never leaves a half-written file at the final path.

    Args:
        dataset: hoopR dataset name, e.g. ``"play_by_play"``. Must be a
            member of :data:`DATASETS`.
        season: Season end year, e.g. 2024 for the 2023-24 season.
        force: When true, re-download even if a valid file already exists at
            the destination. When false (the default), an existing file that
            passes the integrity check is left alone.
        timeout: Per-request timeout in seconds, applied to every HTTP call
            this makes.
        session: HTTP session to issue requests on. A short-lived session is
            created and closed automatically when omitted; callers doing many
            downloads should pass one in to reuse connections (see
            :func:`download_seasons`).

    Returns:
        A :class:`DownloadResult` describing what happened to this one file.

    Raises:
        ValueError: If ``dataset`` is not a recognised hoopR dataset name.
    """
    _validate_dataset(dataset)
    target = season_file("raw", dataset, season, create=True)

    if not force and target.exists() and _is_valid_parquet(target):
        return DownloadResult(dataset=dataset, season=season, status="skipped", path=target)

    if session is not None:
        return _fetch_one(session, dataset, season, target, timeout=timeout)
    with requests.Session() as owned_session:
        return _fetch_one(owned_session, dataset, season, target, timeout=timeout)

download_seasons ¶

download_seasons(dataset, seasons, *, force=False, timeout=_DEFAULT_TIMEOUT_SECONDS, console=None)

Download a range of seasons of one hoopR dataset, reporting progress.

A single HTTP session is reused across the whole range, so the underlying TCP/TLS connection to GitHub is kept warm instead of renegotiated for every file. One season failing does not stop the rest: each season gets its own :class:DownloadResult, so a caller can see exactly which ones need a retry later.

Parameters:

Name	Type	Description	Default
`dataset`	`str`	hoopR dataset name, e.g. `"play_by_play"`. Must be a member of :data:`DATASETS`.	required
`seasons`	`Iterable[int]`	Season end years to fetch, e.g. `range(2015, 2025)` or :data:`PLAY_BY_PLAY_SEASONS`.	required
`force`	`bool`	When true, re-download every season even if a valid file already exists.	`False`
`timeout`	`float`	Per-request timeout in seconds, applied to every HTTP call.	`_DEFAULT_TIMEOUT_SECONDS`
`console`	`Console \| None`	Where to print progress. Defaults to a new :class:`rich.console.Console`.	`None`

Returns:

Name	Type	Description
`One`	`list[DownloadResult]`	class:`DownloadResult` per season, in the order `seasons` was
	`list[DownloadResult]`	iterated.

Raises:

Type	Description
`ValueError`	If `dataset` is not a recognised hoopR dataset name.

Source code in src/pippen/data/hoopr.py

def download_seasons(
    dataset: str,
    seasons: Iterable[int],
    *,
    force: bool = False,
    timeout: float = _DEFAULT_TIMEOUT_SECONDS,
    console: Console | None = None,
) -> list[DownloadResult]:
    """Download a range of seasons of one hoopR dataset, reporting progress.

    A single HTTP session is reused across the whole range, so the underlying
    TCP/TLS connection to GitHub is kept warm instead of renegotiated for
    every file. One season failing does not stop the rest: each season gets
    its own :class:`DownloadResult`, so a caller can see exactly which ones
    need a retry later.

    Args:
        dataset: hoopR dataset name, e.g. ``"play_by_play"``. Must be a
            member of :data:`DATASETS`.
        seasons: Season end years to fetch, e.g. ``range(2015, 2025)`` or
            :data:`PLAY_BY_PLAY_SEASONS`.
        force: When true, re-download every season even if a valid file
            already exists.
        timeout: Per-request timeout in seconds, applied to every HTTP call.
        console: Where to print progress. Defaults to a new
            :class:`rich.console.Console`.

    Returns:
        One :class:`DownloadResult` per season, in the order ``seasons`` was
        iterated.

    Raises:
        ValueError: If ``dataset`` is not a recognised hoopR dataset name.
    """
    _validate_dataset(dataset)
    out = console if console is not None else Console()

    results: list[DownloadResult] = []
    with requests.Session() as session:
        for season in seasons:
            result = download_season(dataset, season, force=force, timeout=timeout, session=session)
            results.append(result)
            out.print(_progress_line(result))
    return results

is_master_dataset ¶

is_master_dataset(dataset)

Whether a dataset is published as one file rather than one per season.

Parameters:

Name	Type	Description	Default
`dataset`	`str`	hoopR dataset name.	required

Returns:

Type	Description
`bool`	True if the dataset has a single master file covering every season.

Source code in src/pippen/data/hoopr.py

def is_master_dataset(dataset: str) -> bool:
    """Whether a dataset is published as one file rather than one per season.

    Args:
        dataset: hoopR dataset name.

    Returns:
        True if the dataset has a single master file covering every season.
    """
    return dataset in _MASTER_FILES

download_master ¶

download_master(dataset, *, force=False, timeout=30.0, session=None)

Download a dataset published as one file covering every season.

Parameters:

Name	Type	Description	Default
`dataset`	`str`	hoopR dataset name, which must be a master dataset. Use :func:`is_master_dataset` to check.	required
`force`	`bool`	Re-download even when a valid file is already present.	`False`
`timeout`	`float`	Per-request timeout in seconds.	`30.0`
`session`	`Session \| None`	HTTP session to reuse. A new one is created if omitted.	`None`

Returns:

Name	Type	Description
`A`	`DownloadResult`	class:`DownloadResult`. Its `season` is 0, since the file covers
	`DownloadResult`	every season and belongs to none of them.

Raises:

Type	Description
`ValueError`	If `dataset` is not a master dataset.

Source code in src/pippen/data/hoopr.py

def download_master(
    dataset: str,
    *,
    force: bool = False,
    timeout: float = 30.0,
    session: requests.Session | None = None,
) -> DownloadResult:
    """Download a dataset published as one file covering every season.

    Args:
        dataset: hoopR dataset name, which must be a master dataset. Use
            :func:`is_master_dataset` to check.
        force: Re-download even when a valid file is already present.
        timeout: Per-request timeout in seconds.
        session: HTTP session to reuse. A new one is created if omitted.

    Returns:
        A :class:`DownloadResult`. Its ``season`` is 0, since the file covers
        every season and belongs to none of them.

    Raises:
        ValueError: If ``dataset`` is not a master dataset.
    """
    if dataset not in _MASTER_FILES:
        known = ", ".join(sorted(_MASTER_FILES))
        raise ValueError(f"{dataset!r} is not published as a master file. Master datasets: {known}")

    stem = _MASTER_FILES[dataset]
    target = dataset_file("raw", dataset, stem, create=True)
    if not force and _is_valid_parquet(target):
        return DownloadResult(dataset=dataset, season=0, status="skipped", path=target)

    url = f"{_RAW_BASE}/{dataset}/{stem}.parquet"
    owned = session is None
    active = session if session is not None else requests.Session()
    try:
        failure = _download_atomically(active, url, target, timeout=timeout)
    finally:
        if owned:
            active.close()

    if failure is not None:
        return DownloadResult(dataset=dataset, season=0, status="failed", reason=failure)
    return DownloadResult(dataset=dataset, season=0, status="downloaded", path=target)

pippen.data.hoopr¶