Set up a personal python package index
Overview
This document describes setting up a private package index as a more convenient way to distribute python packages.
The need to use something like this grows as we add maintainers and packages to maintain.
Our python packages need to be compatile with the standard python package index PyPi.
We use a private package index until we are ready to make a package available to the general public, then we switch to PyPi.
This tool we are using keeps our packages compatible with PyPi for when we do make it available there.
It is a conforming package index system.
We will use it to serve our private package index.
Configuring the SW for devpi
This howto is derived from the following tutorial:
<https://devpi.net/docs/devpi/devpi/stable/+doc/quickstart-releaseprocess.html>_
We are working with python3. We want to setup up a virtual environment that conforms to our assumptions, and then install within that. We call the environment devpi_venv. We create a directory for our setup first.
mkdir ${HOME}/devpi_setup
cd ${HOME}/devpi_setup
We install the bare essentials.
python3 -m venv devpi_venv
source devpi_venv/bin/activate
python -m pip install -U pip
pip install wheel
pip install setuptools
pip install twine
restview is A Restructured Text format viewer I find usefull. (optional)
pip install restview
You invoke it and it keeps re-rendering your file as you modify it.
Like this:
restview README.rst
rst2pdf is convenient for generating pdf copies of .rst documuents.
pip install rst2pdf
rst2pdf doc/privatepypi.rst -o reformatted_docs/privatepypi.pdf
Set up devpi server on our laptop
Install devpi client and web server The non web server - devpi-server - is installed as a consequence.
pip install -U devpi-web devpi-client
Initialize an empty index (at /var/devpi-server by default) Note I had a permissions problem with the init one. So first I manually do this:
sudo mkdir /var/devpi-server
sudo chown ubuntu:ubuntu /var/devpi-server
devpi-init
We want a config file for supervisor daemon to use. This will generate a bunch of config files under the current directory, the one we want included.
cd ${HOME}/devpi_setup
devpi-gen-config
This gives us the file gen-config/supervisor-devpi.conf to copy to /etc/supervisor/conf so that we can start up the server. I started by editing it to say start=False so that it would need to be started up manually. Then I copied it to the right place for it.
sudo cp gen-config/supervisor-devpi.conf /etc/supervisor/conf.d/devpi-server.conf
sudo supervisorctl update
sudo supervisorctl start devpi-server
Create a user, login as him and create the 'dev' index
devpi user -c pbernatchez password=foobar
devpi login pbernatchez --password=foobar
devpi index -c dev bases=root/pypi
Use our dev index
devpi use testuser/dev
Now we can make use of the private index.
We are using flit to publish to our index and it relies on the file : ~/.pypirc.
So we make an entry there for our index.
I gave it the name 'mypypi':
[distutils]
index-servers =
mypypi
testpypi
[mypypi]
repository = http://localhost:3141/pbernatchez/dev
username = pbernatchez
[testpypi]
repository = https://test.pypi.org/legacy/
username = pbernatchez
From here on, using flit, we can refer it as 'mypypi'.
deactivate
cd /home/ubuntu/allrepos/animbboard
source ${HOME}/venv_animbboard/bin/activate
flit build
flit publish --repository mypypi
pip uninstall animbboard
pip install -i http://localhost:3141/pbernatchez/dev animbboard