/ programming

Python Packaging 2017

If you want people to use your code you should package it! You may have heard that packaging is hard but the Python packaging ecosystem has evolved a lot over the years. Taking your beautiful code and sharing it with the world is complex but it doesn't have to be complicated. In this talk you will learn how to take advantage of modern tooling and practices so you can get boring stuff out of the way, publish quickly and frequently, and focus on your code.

  • Dave Forgac (daveops.com)

This post summarizes the topics handled in Dave Forgacs excellent talk given at PyCon 2017. The original blogpost tho the talk can be found here.

I advice you to watch the video first and then come back here to see a writeup of the steps that are involved.

Creating Packages

Use cookiecutter to generate a template structure.

cookiecutter cookiecutter-python-package

It will generate at least something along the following layout (pretty minimal example):

├── MANIFEST.in
├── sample
│   ├── __init__.py
│   ├── package_data.dat
│   └── sample.py
├── README.rst
├── setup.cfg
├── setup.py
└── tests
    ├── __init__.py
    └── test_sample.py

Some notes and recommendations:

  • Commonly the code is stored in an import package with the same name as the package
  • Don't do fancy things in setup.py
  • List required non-python files in Manifest.in
  • Several additional files are commonly seen
    • LICENSE, HISTORY, CONTRIBUTING, AUTHORS
Setting up a new project

Initialize Project and run tests:

  • Set up git repository
  • Add beautiful code
  • Commit and run some tests

Set up Services:

From here on only work in virtualenv !
Use e.g. pew or virtualenvwrapper.

Set up Packaging Tools:
Run pip install wheel and pip install twine

Build Distribution Files:
Run ./setup.py sdist and ./setup.py bdist_wheel or make dist.

Upload:
Use twine: twine upload dist/* as the build-in upload command is insecure and outdated.

Versioning:
Use e.g. bumpversion or versioneer.

Setting up PyPi
  • Register an Account at PyPi and Test PyPi
  • Save PyPi settings in $HOME/.pypirc (password can be left empty if you want to be prompted every time)

[pypi]
repository=https://pypi.python.org/pypi
username=USERNAME
password=PASSWORD
[pypitest]
repository=https://testpypi.python.org/pypi
username=USERNAME
password=PASSWORD

Develop Mode

This will install the package and the dependencies in the current virtualenv, but all changes are reflected immediatly. You will need to reinstall when changing the entry point. The commands are

  • ./setup.py develop or
  • pip install -e .
Setup.py
  • find_packages can be used to help include packages
  • For differences to requirements.txt read this (about 30 minute read).
  • entry_points can be used instead of including binary scripts
Tests and Documentation

If you have tests and documentation, even if only covering a little bit, people are more likely to help extend the coverage. You should always have at least minimal tests and documentations.