Lesson 0002 · Packaging
A built distribution is an archive you can open. By the end of this you will have opened yours and checked its contents against what you intended, instead of trusting that the build did the right thing.
You are about to publish pippen 1.0.0 to PyPI. A version on PyPI
can be deleted but never re-uploaded under the same number, so whatever 1.0.0
contains is what 1.0.0 contains for good. The check below takes about two
minutes and is the last cheap moment to run it.
During the Phase 4 review, one of the questions a reviewer had to answer was whether the wheel and the sdist both carried the computed Parquet tables. That question has a definite answer, obtainable in one command, and this lesson is that command.
An sdist (source distribution) is a .tar.gz of
your project as you wrote it. A wheel is a .zip,
with the extension .whl, of your project as it will be installed.
Installing a wheel is close to unzipping it into site-packages,
which is why wheels install fast and why they replaced the old practice of
running setup.py on the installing machine.
The layouts differ, and the difference is the clearest way to see what each file is for:
| Artifact | Path to the shipped table |
|---|---|
| sdist | pippen-1.0.0/src/pippen/_data/rapm_ratings.parquet |
| wheel | pippen/_data/rapm_ratings.parquet |
The src/ directory exists in your repository to stop Python
importing your package from the working directory instead of from the installed
copy, which is how a test suite ends up passing against code that was never
packaged. It has no meaning after installation, so the wheel does not carry it.
RECORD is the manifestInside a wheel, pippen-1.0.0.dist-info/RECORD lists every file
the wheel installs, each with a hash and a byte count. That file is how
pip uninstall knows precisely which files to remove and no others.
METADATA beside it holds the name, version, dependencies and README
that PyPI renders on your project page.
pippen/py.typed is empty, and emptiness is its entire content.
Under PEP 561, a type checker
analysing someone else's code will ignore your inline annotations unless this
marker is present in the installed package. Its presence is the declaration;
nothing needs to be written inside it. The marker applies recursively, so
shipping it is a promise that the whole package type-checks.
Files that are not .py do not travel by default. In
pyproject.toml the wheel and the sdist are told separately, under
[tool.hatch.build.targets.wheel] and
[tool.hatch.build.targets.sdist]. Telling one and forgetting the
other produces a package that works from a wheel and breaks for anyone who
builds from source, which is a failure that will not show up on your machine.
From the repository root:
uv build
python -c "
import zipfile
z = zipfile.ZipFile('dist/pippen-1.0.0-py3-none-any.whl')
for n in sorted(z.namelist()):
print(f'{z.getinfo(n).file_size:>9,} {n}')
"
Four things to find in the output, each a claim you are checking rather than assuming:
pippen/_data/rapm_ratings.parquet at 235,428 bytespippen/_data/metric_reliability.parquet at 42,049 bytespippen/py.typed at 0 bytesThen the same for the sdist, where the paths carry the src/
prefix and the count is 84:
python -c "
import tarfile
t = tarfile.open('dist/pippen-1.0.0.tar.gz')
for n in sorted(t.getnames()):
print(f'{t.getmember(n).size:>9,} {n}')
"
If a number disagrees with what you expected, that disagreement is the most valuable output this exercise can produce. Bring it to me.
Answer from memory before scrolling back. Retrieving an answer you half remember builds retention in a way that re-reading the paragraph does not.
Why does src/ appear in the sdist and not in the wheel?
The py.typed file in the wheel is zero bytes. What does it do?
What does RECORD in the dist-info directory let pip do?
Inspecting your own artifact is the packaging version of something you already do without thinking in analysis: you do not report a coefficient without looking at the residuals. The build is a transformation, and a transformation you have never looked at the output of is an assumption.
The same move applies to a Docker image (docker run --rm IMAGE ls),
a compiled binary (ldd, nm) and a deployed bundle
(open the source map). In each case the question is identical: the thing that
will run somewhere else, what is actually in it?
Primary source, worth reading in full: Writing your pyproject.toml, from the Python Packaging User Guide. It is maintained by the people who maintain the tooling, and it is the one packaging document that does not go stale, which cannot be said for most Stack Overflow answers on this topic.
Reference sheet: Packaging commands and layouts, for the commands in this lesson without the explanation around them.
Earlier note: 0001: What the data layer taught, which covers atomic writes and validation at the boundary.