Skip to content

pippen.paths

pippen.paths

Resolution of on-disk locations for cached data and build artifacts.

Layout, relative to the data root::

raw/         unmodified downloads, exactly as retrieved
interim/     intermediate tables produced during processing
processed/   analysis-ready tables published in dataset releases
sources/     archived bibliography

The data root is chosen in this order:

  1. the PIPPEN_DATA_DIR environment variable, if set;
  2. a data/ directory beside the repository root, when running from a checkout;
  3. the platform user-cache directory, for installed copies of the package.

data_root 

data_root()

Return the root directory for cached data.

Returns:

Type Description
Path

The resolved data root. The directory is not created by this call.
Source code in src/pippen/paths.py 
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
def data_root() -> Path:
    """Return the root directory for cached data.

    Returns:
        The resolved data root. The directory is not created by this call.
    """
    override = os.environ.get(ENV_VAR)
    if override:
        return Path(override).expanduser().resolve()

    repo = _repo_data_dir()
    if repo is not None:
        return repo

    return Path(user_cache_dir("pippen", appauthor=False))

stage_dir 

stage_dir(stage, *, create=False)

Return the directory for one pipeline stage.

Parameters:

Name Type Description Default
stage str

One of raw, interim, processed or sources.

 required
create bool

When true, create the directory and its parents if missing.

 False

Returns:

Type Description
Path

The resolved stage directory.

Raises:

Type Description
ValueError

If stage is not a recognised stage name.
Source code in src/pippen/paths.py 
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
def stage_dir(stage: str, *, create: bool = False) -> Path:
    """Return the directory for one pipeline stage.

    Args:
        stage: One of ``raw``, ``interim``, ``processed`` or ``sources``.
        create: When true, create the directory and its parents if missing.

    Returns:
        The resolved stage directory.

    Raises:
        ValueError: If ``stage`` is not a recognised stage name.
    """
    if stage not in _STAGES:
        valid = ", ".join(_STAGES)
        raise ValueError(f"unknown stage {stage!r}; expected one of: {valid}")

    path = data_root() / stage
    if create:
        path.mkdir(parents=True, exist_ok=True)
    return path

dataset_file 

dataset_file(stage, dataset, filename, *, create=False)

Return the path for a dataset file that is not split by season.

Some sources publish one file covering every season rather than one file per season. The hoopR schedule is one: a single table spanning 2002 to 2027. Giving it a season-numbered path would mean storing the same file once per season.

Parameters:

Name Type Description Default
stage str

Pipeline stage, as accepted by :func:stage_dir.

 required
dataset str

Dataset name, for example schedules.

 required
filename str

File name without an extension, for example nba_schedule_master.

 required
create bool

When true, create the parent directory if missing.

 False

Returns:

Type Description
Path

The full path to the dataset's Parquet file.
Source code in src/pippen/paths.py 
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
def dataset_file(stage: str, dataset: str, filename: str, *, create: bool = False) -> Path:
    """Return the path for a dataset file that is not split by season.

    Some sources publish one file covering every season rather than one file
    per season. The hoopR schedule is one: a single table spanning 2002 to
    2027. Giving it a season-numbered path would mean storing the same file
    once per season.

    Args:
        stage: Pipeline stage, as accepted by :func:`stage_dir`.
        dataset: Dataset name, for example ``schedules``.
        filename: File name without an extension, for example
            ``nba_schedule_master``.
        create: When true, create the parent directory if missing.

    Returns:
        The full path to the dataset's Parquet file.
    """
    directory = stage_dir(stage) / dataset
    if create:
        directory.mkdir(parents=True, exist_ok=True)
    return directory / f"{filename}.parquet"

season_file 

season_file(stage, dataset, season, *, create=False)

Return the Parquet path for one dataset and one season.

Parameters:

Name Type Description Default
stage str

Pipeline stage, as accepted by :func:stage_dir.

 required
dataset str

Dataset name, for example play_by_play.

 required
season int

Season end year, for example 2024 for the 2023-24 season.

 required
create bool

When true, create the parent directory if missing.

 False

Returns:

Type Description
Path

The full path to the season's Parquet file.
Source code in src/pippen/paths.py 
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
def season_file(stage: str, dataset: str, season: int, *, create: bool = False) -> Path:
    """Return the Parquet path for one dataset and one season.

    Args:
        stage: Pipeline stage, as accepted by :func:`stage_dir`.
        dataset: Dataset name, for example ``play_by_play``.
        season: Season end year, for example 2024 for the 2023-24 season.
        create: When true, create the parent directory if missing.

    Returns:
        The full path to the season's Parquet file.
    """
    directory = stage_dir(stage) / dataset
    if create:
        directory.mkdir(parents=True, exist_ok=True)
    return directory / f"{dataset}_{season}.parquet"